So you push out a piece of code and it eats your live database. The site is broken. You need to take it down to repair the database. So you're going to keep your site down for how long? 30 minutes? 6 hours?
If you're trying to "fix" a database and you're keeping your site down until it's done, get a read-only copy of an old snapshot of the database + site code up. Put up a banner on all pages saying the site is under emergency maintenance so parts of the site are temporarily disabled.
This way your users get to continue using at least the read-only parts of the site and not all of your traffic goes out the window. Keep this in mind when developing the site too; not being able to update a hit counter in the database for a specific page should be a soft error, for example.
If you don't have a place to host this temporary database + site code, think about having such a place. Secondary/failover hosts would work at a time like this, or maybe your single host(s) need more capacity.
Tuesday, August 23, 2011
Wednesday, July 27, 2011
the internet is a collective waste of potential
What really makes me sick about the industry I work in (IT) is how a great majority of the really smart, creative people in it are working on the biggest wastes of time, money and energy on the planet.
Right now, somewhere in the Bay Area, someone is building a tool. In that tool is invested hundreds of thousands, possibly millions, of dollars of resources in human beings' time and the purchase of things to support them. Energy is being expended and people are spending their lives working on this tool. People spent years going to school to amass the knowledge to perform the tasks necessary to complete this tool.
That tool will be used to put funny phrases under pictures of cats on the internet.
Meanwhile, somewhere in sub-Saharan Africa, up to 11 million people may starve to death because they don't have food. When food prices soar (for example, when the USA meddles in food prices to achieve lower cost at the gas pump) and rivers are dry from drought, it hits the poorest the hardest. People's lives are lost as a result.
I'm not someone who fights for causes. I'm as hypocritical, cynical and lazy as most [American] people out there. But I get sick at the thought of the sheer staggering size of waste that is the internet and the big business taking advantage of it. There's untold fortunes of wealth being used to build digital empires who collectively do nothing to help any one or any thing. Sure, facebook creates this big website and eventually people can use it to create an event to rally protesters to a cause. But this was an unintended side-effect, and the end result from such "socializing" is (my guess anyway) ineffective. And before you claim that Facebook is the reason a government is overthrown somewhere in the middle east, please think long and hard about that. Revolution is the domain of people wanting to change something and deciding en masse to put their lives on the line for personal and political freedom. Facebook is the equivalent of a text-based telephone. Do you really think revolution couldn't have happened without a telephone?
My disgust at the waste of potential comes from my experiences in the Open Source community. I noticed how I could spend all my free time working on some cool new toy, only for there to be no real purpose to it. It would languish and if I finished it, nobody would really use it. I noticed how other people tended to spend their time on projects which were fun but produced nothing of value. So I stopped working on things I didn't need. Now I look around and all I can see is wasted effort.
Hackerspaces are one huge example of a waste of resources. Here you have a collective of very smart, motivated inventors who come together - to do what? Create 'makerbots'? Send balloons into space? Build arcade cabinets? WHAT'S THE POINT? You take that same group of people together and ask them to solve something truly difficult - like ways to keep people from dying from starvation in Somalia - and you'd have a real, tangible, valuable product.
Most of the people I know who work to change the world do so in person. I think that's partially because there's more of an immediate gratification and it doesn't take much to fly to Africa and get your hands dirty. But longer-term projects to increase the sustainability of a community are valuable too. You don't have to make huge changes in your life to spend your time working on something of value. All you have to do is change your focus. Do the same job, but pick which employer and project it is based on what kind of value it can produce.
Doing this for your own gratification is a selfish and unseemly objective, to me. If you just want to make yourself feel better you can volunteer at a local homeless shelter. This isn't intended to be a decision based on morals or for some goal to fix the way things are. The goal, to me, is to take the time you spend in life doing "work" and turn it into an investment in the future of the lives of living beings. Because you can spend your time doing nothing - really, it's not hard to do absolutely nothing - or you can spend it doing something which has a positive benefit outside of yourself or the company you work for.
I mean, it's a logical choice... help only yourself, or help yourself and others at the same time. In our society we do for others all the time because doing good things is cyclical. We can eat because we pay people to create and bring us food instead of stealing it (just ask warlords; it's not a sustainable business model). We don't get murdered because we don't murder people. And we hold doors for people so they too will hold a door for us. In this way, creating something of value which provides for other people will improve society - and on a bigger scale, the world. If you figure out a way to keep people from going hungry, we don't need to spend billions on foreign aid, which strengthens our economy is stronger. It's a simplistic but effective idea.
The next time you're considering job offers or personal projects to pick up, ask what the end result of the work is. If one answer is "put funny phrases on pictures of cats", and the other is "helping people", consider the second one. It may benefit you more in the end.
Right now, somewhere in the Bay Area, someone is building a tool. In that tool is invested hundreds of thousands, possibly millions, of dollars of resources in human beings' time and the purchase of things to support them. Energy is being expended and people are spending their lives working on this tool. People spent years going to school to amass the knowledge to perform the tasks necessary to complete this tool.
That tool will be used to put funny phrases under pictures of cats on the internet.
Meanwhile, somewhere in sub-Saharan Africa, up to 11 million people may starve to death because they don't have food. When food prices soar (for example, when the USA meddles in food prices to achieve lower cost at the gas pump) and rivers are dry from drought, it hits the poorest the hardest. People's lives are lost as a result.
I'm not someone who fights for causes. I'm as hypocritical, cynical and lazy as most [American] people out there. But I get sick at the thought of the sheer staggering size of waste that is the internet and the big business taking advantage of it. There's untold fortunes of wealth being used to build digital empires who collectively do nothing to help any one or any thing. Sure, facebook creates this big website and eventually people can use it to create an event to rally protesters to a cause. But this was an unintended side-effect, and the end result from such "socializing" is (my guess anyway) ineffective. And before you claim that Facebook is the reason a government is overthrown somewhere in the middle east, please think long and hard about that. Revolution is the domain of people wanting to change something and deciding en masse to put their lives on the line for personal and political freedom. Facebook is the equivalent of a text-based telephone. Do you really think revolution couldn't have happened without a telephone?
My disgust at the waste of potential comes from my experiences in the Open Source community. I noticed how I could spend all my free time working on some cool new toy, only for there to be no real purpose to it. It would languish and if I finished it, nobody would really use it. I noticed how other people tended to spend their time on projects which were fun but produced nothing of value. So I stopped working on things I didn't need. Now I look around and all I can see is wasted effort.
Hackerspaces are one huge example of a waste of resources. Here you have a collective of very smart, motivated inventors who come together - to do what? Create 'makerbots'? Send balloons into space? Build arcade cabinets? WHAT'S THE POINT? You take that same group of people together and ask them to solve something truly difficult - like ways to keep people from dying from starvation in Somalia - and you'd have a real, tangible, valuable product.
Most of the people I know who work to change the world do so in person. I think that's partially because there's more of an immediate gratification and it doesn't take much to fly to Africa and get your hands dirty. But longer-term projects to increase the sustainability of a community are valuable too. You don't have to make huge changes in your life to spend your time working on something of value. All you have to do is change your focus. Do the same job, but pick which employer and project it is based on what kind of value it can produce.
Doing this for your own gratification is a selfish and unseemly objective, to me. If you just want to make yourself feel better you can volunteer at a local homeless shelter. This isn't intended to be a decision based on morals or for some goal to fix the way things are. The goal, to me, is to take the time you spend in life doing "work" and turn it into an investment in the future of the lives of living beings. Because you can spend your time doing nothing - really, it's not hard to do absolutely nothing - or you can spend it doing something which has a positive benefit outside of yourself or the company you work for.
I mean, it's a logical choice... help only yourself, or help yourself and others at the same time. In our society we do for others all the time because doing good things is cyclical. We can eat because we pay people to create and bring us food instead of stealing it (just ask warlords; it's not a sustainable business model). We don't get murdered because we don't murder people. And we hold doors for people so they too will hold a door for us. In this way, creating something of value which provides for other people will improve society - and on a bigger scale, the world. If you figure out a way to keep people from going hungry, we don't need to spend billions on foreign aid, which strengthens our economy is stronger. It's a simplistic but effective idea.
The next time you're considering job offers or personal projects to pick up, ask what the end result of the work is. If one answer is "put funny phrases on pictures of cats", and the other is "helping people", consider the second one. It may benefit you more in the end.
Tuesday, July 12, 2011
Social Media Hoedown
Honestly. The fact that people haven't quite grasped that social media is all about fads is a little scary to me. You don't really have to a deliver a 'product' as anything other than a slick UI that allows people to play with each other. It's communication for entertainment's sake.
There is no point to using social media other than event invites, relationship status and pictures. That's the only useful features. Well, and contact information, but you've already got their contact info if they're really your friend.
I know, I know. You're going to defend your meek social interaction through comments and statuses and links and videos and all kinds of other nonsense. We've had forums for years. Some people make friends on forums but they're going to stick to the forums, not their facebook.
Google+ is just the latest reincarnation of the social media supersite. After them will be another. It doesn't matter to anyone what site they use as long as it's new and it's slicker. Why do they not care? Because there's no value in it besides the 3 things I mentioned above. As long as everyone they know is on the site, they'll use it.
So what's the killer product nobody's made yet? Quite simply it's a service that integrates every communication medium that people use. If (for example) you had a deal with every major wireless carrier to carry your apps and optimized communication through each varying protocol (SMS, SMTP, voice, HTTP, etc) to allow seamless and instant communication, there'd be no reason not to use it. If nobody ever had to sign up to a service because they were instantly and intrinsically enjoined with it there'd be nothing much else to sway one's opinion (besides Farmville). Nor would you have a choice, really.
And that's not to put down Farmville: Mindless games and apps have huge value for their market, but you don't need a social media network for that. Moreover, this seamless communication medium would allow you a framework to build apps which could reach anyone anywhere. Combine the dedicated carrier apps with a means to ship targeted "value-adding" applications and you've got one powerful, flexible social engine.
The way I see it is, all of these "sites" are based on some archaic notion that people should be using "the web" to get what it is they want. I disagree. I see every device with a network stack as simply a means to an end. The ends are basic: communication, information/entertainment and acquiring of goods/services. You can do all of those things with SMTP and POP3 if the sent and received messages are tailored for the application.
So let's unburden ourselves from the chains of some complex and limiting set of protocols and scripting languages. Nobody *needs* an app or a site. What we need are practical multifaceted interfaces to basic human interaction.
Google+ isn't going to give us that. The next site that replaces Google+ when the hoedown continues and the winds change, also won't give us that. But maybe once we've wasted enough time playing with our toys we'll finally get tired enough to just make tools that give us what we need and not always what we think we want.
There is no point to using social media other than event invites, relationship status and pictures. That's the only useful features. Well, and contact information, but you've already got their contact info if they're really your friend.
I know, I know. You're going to defend your meek social interaction through comments and statuses and links and videos and all kinds of other nonsense. We've had forums for years. Some people make friends on forums but they're going to stick to the forums, not their facebook.
Google+ is just the latest reincarnation of the social media supersite. After them will be another. It doesn't matter to anyone what site they use as long as it's new and it's slicker. Why do they not care? Because there's no value in it besides the 3 things I mentioned above. As long as everyone they know is on the site, they'll use it.
So what's the killer product nobody's made yet? Quite simply it's a service that integrates every communication medium that people use. If (for example) you had a deal with every major wireless carrier to carry your apps and optimized communication through each varying protocol (SMS, SMTP, voice, HTTP, etc) to allow seamless and instant communication, there'd be no reason not to use it. If nobody ever had to sign up to a service because they were instantly and intrinsically enjoined with it there'd be nothing much else to sway one's opinion (besides Farmville). Nor would you have a choice, really.
And that's not to put down Farmville: Mindless games and apps have huge value for their market, but you don't need a social media network for that. Moreover, this seamless communication medium would allow you a framework to build apps which could reach anyone anywhere. Combine the dedicated carrier apps with a means to ship targeted "value-adding" applications and you've got one powerful, flexible social engine.
The way I see it is, all of these "sites" are based on some archaic notion that people should be using "the web" to get what it is they want. I disagree. I see every device with a network stack as simply a means to an end. The ends are basic: communication, information/entertainment and acquiring of goods/services. You can do all of those things with SMTP and POP3 if the sent and received messages are tailored for the application.
So let's unburden ourselves from the chains of some complex and limiting set of protocols and scripting languages. Nobody *needs* an app or a site. What we need are practical multifaceted interfaces to basic human interaction.
Google+ isn't going to give us that. The next site that replaces Google+ when the hoedown continues and the winds change, also won't give us that. But maybe once we've wasted enough time playing with our toys we'll finally get tired enough to just make tools that give us what we need and not always what we think we want.
Monday, June 20, 2011
An Exercise In Fear: Why We Care About A Bunch Of 15 Year Old Retards
If you know anything about LulzSec it's that its members are (or were until recently) 4chan users, probably of the /b/ variety. Everything from their namesake to their cause to their speech and online habits pretty much says /b/tard.
I know /b/tards. Some of them are nice people (though most are dead inside). I basically get why they are on there, looking at posts of dead people and stupid unfunny cartoons and fag jokes and the inevitable hentai bestiality incest child rape porn. It's because they're bored. They're bored and so they go on the internet to find something to entertain them. And they find lots of other really bored people who like to look at shocking things and basically be idiots. That's the whole reason for 4chan. People are just horrible, and that's why that's there.
Not that i'm complaining. I grew up on the internet. I've looked at and read every horrible despicable thing the human imagination can think up. So i'm not harboring any grudge or ill will against these people. But I think i've gotten to the point where i'm sick of looking at boring, mindless, stupid shit. Unfortunately I can't completely ignore it because of LulzSec and Anonymous.
Why is the media giving so much attention to whatever crap LulzSec decides to announce? Today on Google News one of the top stories was the same story I had read on Hacker News: LulzSec decides to go on some "new mission" wherein they will attempt to deface government websites. Do you realize how completely boring that is? Do you know how much of a fucking loser you have to be to dedicate your valuable time to erasing a web page? The fact that just this announcement was news worthy makes one thing clear: people are fascinated and afraid of LulzSec.
The attacks carried out in their name have been many and they have infiltrated some very large and incredibly, stupidly insecure sites. The subsequent release of information from these sites has been absurdly large. On top of that, they command a sizeable botnet with which they DDoS whoever the fuck they feel like at the moment.
Are these attacks 'sophisticated'? No. There are many freely available tools which can be used to automate looking for and exploiting holes in public web applications and network services. Botnets are also not very hard to 'get'; most botnet owners don't properly secure their botnets and many can simply be social engineered to hand over control of the botnet. Most security researchers i've talked to don't find much difficulty in acquiring tens of thousands of nodes.
However, these tools are effective. Clearly there are many large sites with old holes waiting to be taken advantage of, and a DDoS is a very effective means of taking a host offline if you don't have the skill to penetrate it. Thus they can and do cause quite a bit of mischief. But why are we getting a news bulletin every time they do some damage?
Ultimately we are playing into their media-whoring hands. A couple of kids who are really bored are finding lots of attention (both positive and negative) in creating havoc on the internet. With each site taken down and subsequent press release they get more infamous and thus the next attack or announcement gets even more press. Online businesses cower in fear waiting for the next attack, and when it affects users directly (like the many gamers affected by their DDoSing) they are sucked into a whirlpool of hate directed at LulzSec - who, being 4chan trolls, revel in the fact that they could make such a large user base 'mad'.
Where do we go from here? Do we attempt to ignore the internet bullies in the hopes that they'll go away? Do we attack back and start a ridiculous arms race of morons flinging poo at each other? Should the media stop giving them a loudspeaker, or should it try instead to exercise some investigative journalism instead of parroting their exploits?
The truth is that people are simple. LulzSec will keep this up for a little longer, looking for big targets to attack to remain media darlings. We'll keep eating it up because people like celebrity gossip. But for the most part, everything will be the same as it always has been.
The difference is that now there's an 800lb gorilla in the room exposing the horribly lax security practices some of us know to be standard fair in the corporate IT world. Perhaps we'll get some tough new laws and a prison sentence to try to discourage this type of behavior in the future. If there's a positive effect of this whole episode it's that we can use LulzSec as bogey men to scare developers and sysadmins into doing their due diligence to keep their systems secure.
But then, when the lights go down and the circus is over, everything will go back to the way it was, and we'll sleep soundly until another bunch of bored teens decide to DDoS or exploit another service. Hopefully we can prevent this kind of thing from happening again by just not playing into the trolls' hands.
I know /b/tards. Some of them are nice people (though most are dead inside). I basically get why they are on there, looking at posts of dead people and stupid unfunny cartoons and fag jokes and the inevitable hentai bestiality incest child rape porn. It's because they're bored. They're bored and so they go on the internet to find something to entertain them. And they find lots of other really bored people who like to look at shocking things and basically be idiots. That's the whole reason for 4chan. People are just horrible, and that's why that's there.
Not that i'm complaining. I grew up on the internet. I've looked at and read every horrible despicable thing the human imagination can think up. So i'm not harboring any grudge or ill will against these people. But I think i've gotten to the point where i'm sick of looking at boring, mindless, stupid shit. Unfortunately I can't completely ignore it because of LulzSec and Anonymous.
Why is the media giving so much attention to whatever crap LulzSec decides to announce? Today on Google News one of the top stories was the same story I had read on Hacker News: LulzSec decides to go on some "new mission" wherein they will attempt to deface government websites. Do you realize how completely boring that is? Do you know how much of a fucking loser you have to be to dedicate your valuable time to erasing a web page? The fact that just this announcement was news worthy makes one thing clear: people are fascinated and afraid of LulzSec.
The attacks carried out in their name have been many and they have infiltrated some very large and incredibly, stupidly insecure sites. The subsequent release of information from these sites has been absurdly large. On top of that, they command a sizeable botnet with which they DDoS whoever the fuck they feel like at the moment.
Are these attacks 'sophisticated'? No. There are many freely available tools which can be used to automate looking for and exploiting holes in public web applications and network services. Botnets are also not very hard to 'get'; most botnet owners don't properly secure their botnets and many can simply be social engineered to hand over control of the botnet. Most security researchers i've talked to don't find much difficulty in acquiring tens of thousands of nodes.
However, these tools are effective. Clearly there are many large sites with old holes waiting to be taken advantage of, and a DDoS is a very effective means of taking a host offline if you don't have the skill to penetrate it. Thus they can and do cause quite a bit of mischief. But why are we getting a news bulletin every time they do some damage?
Ultimately we are playing into their media-whoring hands. A couple of kids who are really bored are finding lots of attention (both positive and negative) in creating havoc on the internet. With each site taken down and subsequent press release they get more infamous and thus the next attack or announcement gets even more press. Online businesses cower in fear waiting for the next attack, and when it affects users directly (like the many gamers affected by their DDoSing) they are sucked into a whirlpool of hate directed at LulzSec - who, being 4chan trolls, revel in the fact that they could make such a large user base 'mad'.
Where do we go from here? Do we attempt to ignore the internet bullies in the hopes that they'll go away? Do we attack back and start a ridiculous arms race of morons flinging poo at each other? Should the media stop giving them a loudspeaker, or should it try instead to exercise some investigative journalism instead of parroting their exploits?
The truth is that people are simple. LulzSec will keep this up for a little longer, looking for big targets to attack to remain media darlings. We'll keep eating it up because people like celebrity gossip. But for the most part, everything will be the same as it always has been.
The difference is that now there's an 800lb gorilla in the room exposing the horribly lax security practices some of us know to be standard fair in the corporate IT world. Perhaps we'll get some tough new laws and a prison sentence to try to discourage this type of behavior in the future. If there's a positive effect of this whole episode it's that we can use LulzSec as bogey men to scare developers and sysadmins into doing their due diligence to keep their systems secure.
But then, when the lights go down and the circus is over, everything will go back to the way it was, and we'll sleep soundly until another bunch of bored teens decide to DDoS or exploit another service. Hopefully we can prevent this kind of thing from happening again by just not playing into the trolls' hands.
Wednesday, June 1, 2011
devops/deveng is still a bad idea
I'm sure in 10 years people will eventually get just why it's a crappy idea to reinvent the wheel in Operations departments. Maybe somebody will finally standardize on a set of software configurations to deploy to manage an enterprise network. Could just be wishful thinking, but you never know.
Right now I work at a large non-profit as a small cog that's part of some big wheels. There's lots of people on my team that all basically do one or two specific jobs. In terms of how we go about accomplishing tasks it's incredibly inefficient. But that's the nature of big non-profits, I assume.
One example of how this "devops" idea fails came up recently. Somebody wanted a tool that rotates logs passed via standard-in. This is an old problem: you've got to keep processing logs but you can't afford the downtime of moving the log and restarting the service. So you open a pipe to a program which handles the juggling of logs for you. The easy choice is cronolog, a very old, stable open source tool.
When I asked if we could install it on all our RHEL boxes, I was told there was too much red tape involved with just getting a 'yum install' done, so I should just compile and install the software locally. (Yeah, i'm serious.) So after doing this I modify the script we need to use cronolog, test it, and it works great as expected.
Once i'm ready to push this out everywhere i'm told to hold off, that we need a better solution. Better than 'its done and working'? It turns out, if you use 3rd party software like this in our environment, three things happen:
1. Since we don't pay for the software there's no support contract. If there's no support, apparently it's too much trouble to find somebody who knows C in a building full of developers to support the software.
2. As a side-effect of #1, the security team can't get updates from a vendor and thus might not allow us to use the software as it might have unknown security holes.
3. We have to change our build standard to include the new software.
Now keep in mind, cronolog is shipped with RHEL. It just isn't installed on the machines. So getting it installed brings all sorts of red tape questions. What's their solution? Write something from scratch in-house. This of course is a lovely paradox, because:
1. We have to support the in-house solution now.
2. Nobody is going to audit the in-house solution for security holes.
Of course we don't have to change our build standard and it isn't a security conflict for one simple reason: Everyone ignores in-house software. That's right: the loophole to the regulation is to simply completely ignore auditing for internal tools. And there's an interesting point about devops/engops.
If you had actually paid for a product instead of developing it in-house or getting an open source solution, you would have assurance of both the attention to and patching for security holes, and you would just use it - there's no custom scripting or in-house wrapping required. Not only does it speed up your work to pay for it but it makes it more secure (in a fashion) and prevents you from having to spend development cycles.
The next time you decide to hire a 'devop' or 'deveng', consider how much money and time you'd save if you just spent a couple hundred or grand on completed, supported tools. (And as an aside, try not to allow gaping loopholes in logic like the ones pointed out above)
Edit: my mistake, cronolog is not shipped with RHEL (why I do not know). I ended up writing my own as it's much simpler than I originally thought. However, getting them to support 'logrotate' on solaris is looking like a bigger challenge...
Right now I work at a large non-profit as a small cog that's part of some big wheels. There's lots of people on my team that all basically do one or two specific jobs. In terms of how we go about accomplishing tasks it's incredibly inefficient. But that's the nature of big non-profits, I assume.
One example of how this "devops" idea fails came up recently. Somebody wanted a tool that rotates logs passed via standard-in. This is an old problem: you've got to keep processing logs but you can't afford the downtime of moving the log and restarting the service. So you open a pipe to a program which handles the juggling of logs for you. The easy choice is cronolog, a very old, stable open source tool.
When I asked if we could install it on all our RHEL boxes, I was told there was too much red tape involved with just getting a 'yum install' done, so I should just compile and install the software locally. (Yeah, i'm serious.) So after doing this I modify the script we need to use cronolog, test it, and it works great as expected.
Once i'm ready to push this out everywhere i'm told to hold off, that we need a better solution. Better than 'its done and working'? It turns out, if you use 3rd party software like this in our environment, three things happen:
1. Since we don't pay for the software there's no support contract. If there's no support, apparently it's too much trouble to find somebody who knows C in a building full of developers to support the software.
2. As a side-effect of #1, the security team can't get updates from a vendor and thus might not allow us to use the software as it might have unknown security holes.
3. We have to change our build standard to include the new software.
Now keep in mind, cronolog is shipped with RHEL. It just isn't installed on the machines. So getting it installed brings all sorts of red tape questions. What's their solution? Write something from scratch in-house. This of course is a lovely paradox, because:
1. We have to support the in-house solution now.
2. Nobody is going to audit the in-house solution for security holes.
Of course we don't have to change our build standard and it isn't a security conflict for one simple reason: Everyone ignores in-house software. That's right: the loophole to the regulation is to simply completely ignore auditing for internal tools. And there's an interesting point about devops/engops.
If you had actually paid for a product instead of developing it in-house or getting an open source solution, you would have assurance of both the attention to and patching for security holes, and you would just use it - there's no custom scripting or in-house wrapping required. Not only does it speed up your work to pay for it but it makes it more secure (in a fashion) and prevents you from having to spend development cycles.
The next time you decide to hire a 'devop' or 'deveng', consider how much money and time you'd save if you just spent a couple hundred or grand on completed, supported tools. (And as an aside, try not to allow gaping loopholes in logic like the ones pointed out above)
Edit: my mistake, cronolog is not shipped with RHEL (why I do not know). I ended up writing my own as it's much simpler than I originally thought. However, getting them to support 'logrotate' on solaris is looking like a bigger challenge...
Monday, March 28, 2011
the real problem and real solution for https
(for background see this hackerne.ws post)
Really, the problem here is nobody trusts the CAs. (it is kind of difficult to just assume 650 different CAs will all maintain 100% security over their cert-generation procedures)
What people seem to be looking for here is an early warning system for possible mitm *after* an initial "trusted" connection (which you can never tell for sure because even if you're browsing from a secure LAN the destination site could have been compromised, or the nameserver).
The best solution to this problem is one that will fix the problem of a trusted initial connection without relying on CAs. Of course this is difficult. But let's simplify it a bit first.
Assume for a second the internet is only two nodes: me at a desk in one room and an ethernet cable that goes into an adjoining room. I don't know who's in that other room - but I want to connect to the other end and do my banking. So how do I know if it's my bank in there or a stranger?
What do I have that I can trust right now? I have my computer, which contains (among other things) an operating system and a web browser. We won't discuss how it got there, because to discuss the origin of things ad infinitum will leave us with religion and that never solves anything.
My browser already ships with trusted information: the certificates of trusted authorities. But it's trying too hard to make everybody happy, sucking up as many different sources of trusted information as possible. In ten years we might have 6,500 trusted CAs. This won't end well.
I just want to tell if the person in the room is really my bank. What can I do to be sure?
I can ask the person in the room a secret only my bank knows, for one thing. I could have told my bank in person or over the phone, via a letter, or probably whenever I joined my bank for the first time. This would be something only I would know, not something that came with my computer.
Another way I can be sure is if I verify all the steps in the process to connect to my bank. If someone was trying to fake being the bank, they would probably have a different IP address and a different certificate. If I had a print-out with a hash of those and punched it into my browser before I made the connection, my browser would know for sure this is the bank's information and reject the connection if anything was different.
What we have here essentially is static configuration and CHAP. Combine this with PKI and you have three separate pieces of information which must be spoofed in order to successfully compromise the connection. If the person in the other room knows all of this, the bank is surely compromised anyway.
Note that this is merely a way to establish a "secure first connection". You will still need to at least have specific information about the site you're connecting to (IP and cert hash) and to be more sure of the connection have already registered and set a secret only you and the other person know.
There are technical limitations with some of this design, but if these can be worked around you should have a fairly secure connection. If you want to sleep better about PKI there should be less CAs (more like 10 instead of 650).
Really, the problem here is nobody trusts the CAs. (it is kind of difficult to just assume 650 different CAs will all maintain 100% security over their cert-generation procedures)
What people seem to be looking for here is an early warning system for possible mitm *after* an initial "trusted" connection (which you can never tell for sure because even if you're browsing from a secure LAN the destination site could have been compromised, or the nameserver).
The best solution to this problem is one that will fix the problem of a trusted initial connection without relying on CAs. Of course this is difficult. But let's simplify it a bit first.
Assume for a second the internet is only two nodes: me at a desk in one room and an ethernet cable that goes into an adjoining room. I don't know who's in that other room - but I want to connect to the other end and do my banking. So how do I know if it's my bank in there or a stranger?
What do I have that I can trust right now? I have my computer, which contains (among other things) an operating system and a web browser. We won't discuss how it got there, because to discuss the origin of things ad infinitum will leave us with religion and that never solves anything.
My browser already ships with trusted information: the certificates of trusted authorities. But it's trying too hard to make everybody happy, sucking up as many different sources of trusted information as possible. In ten years we might have 6,500 trusted CAs. This won't end well.
I just want to tell if the person in the room is really my bank. What can I do to be sure?
I can ask the person in the room a secret only my bank knows, for one thing. I could have told my bank in person or over the phone, via a letter, or probably whenever I joined my bank for the first time. This would be something only I would know, not something that came with my computer.
Another way I can be sure is if I verify all the steps in the process to connect to my bank. If someone was trying to fake being the bank, they would probably have a different IP address and a different certificate. If I had a print-out with a hash of those and punched it into my browser before I made the connection, my browser would know for sure this is the bank's information and reject the connection if anything was different.
What we have here essentially is static configuration and CHAP. Combine this with PKI and you have three separate pieces of information which must be spoofed in order to successfully compromise the connection. If the person in the other room knows all of this, the bank is surely compromised anyway.
Note that this is merely a way to establish a "secure first connection". You will still need to at least have specific information about the site you're connecting to (IP and cert hash) and to be more sure of the connection have already registered and set a secret only you and the other person know.
There are technical limitations with some of this design, but if these can be worked around you should have a fairly secure connection. If you want to sleep better about PKI there should be less CAs (more like 10 instead of 650).
Thursday, March 3, 2011
universal webapp architecture
I think its funny when people have to redesign their architecture. It's like, what, you couldn't scale? Just throw some hardware at it.
Your framework had limits? Why'd you reuse it in a way that would eventually run into bottlenecks? Didn't you learn how the whole thing worked before deciding to implement it?
Your code is slowing down and bloating up, so you figure a redesign is easier than optimization? Congratulations, you've fallen victim to the worst thing you can do when faced with performance problems: throwing the baby out with the bath water.
Just optimize your current crappy system and add layers to make it get cheap performance, scale horizontally and get on with business. Redesign is usually a waste of business resources.
Your framework had limits? Why'd you reuse it in a way that would eventually run into bottlenecks? Didn't you learn how the whole thing worked before deciding to implement it?
Your code is slowing down and bloating up, so you figure a redesign is easier than optimization? Congratulations, you've fallen victim to the worst thing you can do when faced with performance problems: throwing the baby out with the bath water.
Just optimize your current crappy system and add layers to make it get cheap performance, scale horizontally and get on with business. Redesign is usually a waste of business resources.
Subscribe to:
Posts (Atom)