how to build a high-traffic website

this is all common sense information based on some simple principles. i am not an engineer, but i feel i understand what makes things fast or slow in general and how to apply them in a world of software and enterprise systems.

let's start with cars. something light is going to be faster than something heavier, given the same horsepower. if it's lighter it takes less force to move it. you can compare this to computing as anything 'light' in memory use, disk operations or cpu cycles will take less time to finish 'computing.' keep everything light so you can scale 1000x at a moment's notice.

next, you need your 'frame' to be strong enough to handle the horsepower you throw at it. sheet metal and aluminum will fold up pretty quickly when the torque of 500 horses jars and twists it all at once. this applies not only to the framework of your software, but to the framework of your systems and your network which your systems operate on. they need to scale infinitely (yes, as in to infinity) and they need to be built to handle all the future needs/requirements. when you try to add a hack or reinvent your architecture later you'll find yourself in quite a tight spot, and it will be better if you can just move your existing resources around to handle additional load.

of course, if a car is exorbitantly fast you'll need lots of safety to keep all that power and speed in check. this can mean redundancy and also testing/integrity checking. first you need to consider your technology: are you using Hadoop? if so, you've got a big gaping single-point-of-failure you need to ensure never goes down or is always able to be replicated or hot-failed quickly. what happens when any given part of your infrastructure goes down? will it cause the whole thing to come to a screeching halt or will your car still drive OK with just 7 cylinders? you'll also need to write tests for your software if it changes frequently to ensure it's operating as expected. you could also do to implement a configuration management and tripwire solution to track any change on your systems and apply the correct good working configuration. this isn't a security requirement, this is essential to just keeping everything running.

next, you'll need lots of dials, knobs and lights to know everything is running smoothly. when you have a race car or even a humble sports car, you need to know your oil is OK, your fuel is plentiful and that your engine isn't knocking. this means lots and lots of stats and graphs and monitors. more than you could ever need. you don't want to be caught with your pants down and your car stalled out in the middle of the track because you weren't watching the oil temp level. you need to make sure your disks aren't filling up and know before they do. you need to know the *rate* at which your resources are being used so you have ample warning before things get too close to fix. you need to know when some service is down, and if at all possible get it back up automatically.

finally: the car's performance is dependent upon its mechanics and engineers. you need the people working on it to know what they're doing and you have to make sure they're doing their job right, or you could end up with a catastrophe. you can't afford to have a developer making sql queries that take 50 seconds to complete. you also can't afford to have a sysadmin making local edits or ignoring system faults, or really anybody to falter on their layer. you *need* to police everyone and make sure everything is done *right* with no shortcuts or bad designs. you cannot ignore the fact that diligence will always result in a more reliable system than those without.