solaris admins are masochists

it's been a while since i had to do active development on solaris boxes. god they suck.

1. tar doesn't natively handle compression and doesn't do simple things like "tar -xvf foo.tar" right (probably trying to extract to / by default).
2. the default shell is not bash.
3. ls doesn't understand simple things like `ls DIRECTORY -la` (change order of options) and it doesn't do color (supposedly because some retarded admins think a more optimal user interface is 'unprofessional').
4. you have to fight with ps to get it to list anything the way you want unless you use -o and hope some other option you used doesn't break it.
5. trying to figure out how much memory you have is a nightmare.
6. most system information is complex and hidden behind solaris-specific tools or APIs.
7. /export/home and /home exists (sometimes on different partitions), and for some reason root's home directory is / even though /root exists.
8. the OS does nothing to set useful values like $PS1.

all that is just stuff that's happened to me within 5 minutes. jesus christ solaris, you've had a million years to catch up to usability of gnu tools. either replace your old broken shit or freshen it up. (while people still use your antiquated crap)

reality inside a video game

the thought occurred to me that in virtual worlds, avatars rarely (if ever) age. as far as i can tell, The Sims 2 is the only game where characters are born, grow old, and subsequently die. i think this paints a vivid picture of how we see our virtual worlds and how we wish to spend our time - that is to say, not thinking about mortality.

probably most video games which involve a [virtually] living entity involve certain properties of immortality. usually an avatar will find it difficult to die, and once they do die, are immediately given the opportunity to come back to life. it's left up to the user to determine if they should be brought back or not. they can be revived indefinitely if the user so chooses.

but where's the reality in that? there is no 'save point' in real life. if the user dies, it's game over permanently. in many aspects we have total control over our lives and in a very GTA way we can do anything we want. but we also have our own personal limitations and the limitations of the world around us. we are free to expand, only to be confined in boundaries.

so where is the birth? where is the growth, the learning, the adaptation and choices that shape our lives? why haven't we fully grasped those crucial factors of real life and distilled them to a video game form? to me, this is the ultimate video game: one in which just playing the game changes the shape and course of our lives. where we're no longer bystanders but active participants.

in my game the user gets only one avatar, and they have to follow it through its life. if the avatar dies, they don't get to play again until the average life expectancy of that character is up. there will be no way to circumvent this - no "ruling class" shaped by how much money the user pumps into the system to try to revive their character. everyone gets the same access and plays by the same rules. there will also be no "gold farming", no multiple accounts, trading or buying/selling. the ability to play this game will be rigidly structured, with the same building blocks, freedoms and boundaries as the user could find in our own world. this will be the most addictive game ever made.

obviously with such imposing limits on how and if the user can play, people will have to be very careful how they play. there will be no running around like a jack-ass and fucking with people, because who knows... they might just stab the user for being such a jack-ass. now he/she's dead, and they can't play again for 40 or 50 literal human years. kind of a morbid warning to others, but it reflects some of the motivations we all have to remain calm, respectful human beings instead of what we regress to on anonymous mediums without fear of retribution.

besides this "real reality" imposed by the limitations and fragility of an existence such as human beings, users will also experience what it is like to grow up inside the system. they'll start either from birth or from being very small children, perhaps with 'parents' or some other adult guardian figure. the hope is that we can actually teach people lessons about life. obviously most users will already have the knowledge of a teenager or older, so basic concepts like reading, history, etc may not be necessary. but the interaction with other avatars - also controlled by users jsut like them, in an environment not unlike the real world - may help them realize things about life they hadn't noticed before. in this world the user isn't the same person - they are someone unique, and they don't get to decide who they are. by living life in the shoes of a random individual they may find new things to discover about the real world. in this way, the user actually learns as the avatar does.

this whole concept hinges on the idea of imposing all the restrictions of the real world in the virtual one. an avatar cannot be allowed to have anything given to them except what they may receive as a normal part of trade or the economy of the virtual world. users may not "inject" currency into the system - as in the real world, they have to earn, steal or be gifted anything they want or need.

this brings up some more real-life aspects not often found in video games (except perhaps the Sims): food, clothing, shelter. we all need it, and it's almost never for free. unless someone decides to build a homeless shelter and find a way to gather the resources to give this stuff away for free, everything is simply acquired via the standard means used in the real world. but the avatar must eat, and must sleep, and must be kept in good health. all of these things are related to how we interact with our real world and how we live and grow and learn, and thus must be replicated in the virtual world.

as in the real world, there will be ways for avatars to go to school, read books, listen to music, play basketball... anything we can think of to try to replicate the common human experience. but this also includes the negative aspects of our human condition. selling drugs, molesting children, murdering, overthrowing governments. oh yes, we'll need a government and people to enforce its laws. i think it will be interesting to start with anarchy, and see if a system of government (or perhaps a cult?) develops to enforce rule of law and order. because many people will be apathetic to the idea of this virtual world, many people may begin playing and immediately try to wreak havoc. this is part of how starting as a child may be beneficial; they may not be able to attain the resources to cause much trouble. but the people will also need to police themselves and try to prevent malice from destroying what they've built for themselves.

the idea is difficult to realize. to build an entire real world in a computer... is this not what The Matrix is supposed to be? it seems like an enormous undertaking, and one fraught with trouble. but as we learned from The Matrix, there must be flaws for it to be convincing. it must be a harsh world if people are going to recognize it as real. however, this world will have no AI (until at some point the scale of the system requires it). any AI that people could actually interact with would defeat the purpose of the entire system. in a realistic world, we build it, we shape it. there may be things about nature that help define what it is, but ultimately we determine our own destinies. an AI mucking things up would be like a God that gets to decide if you get ice cream or don't get ice cream. in truth, we all know it is us that makes such decisions.

how i browse the web

if i were really smart i'd set up SELinux and use sandbox -X to create a secure sandbox for my Firefox and other networked applications. i still want to get around to this one day, but it seems too complicated for me to learn how to write all the rules necessary in 30 minutes.

instead i use a combination of tunneling and secure filtering to lock down my browsing session. first of all, all my traffic goes through an ssh SOCKS tunnel to a VPS i pay for (these can be as little as $4 with a 100GB or more bandwidth cap, so more than enough for general browsing needs). this immediately solves the "starbucks sniffer" problem, and thus my only worry left is the traffic from my VPS to the websites i'm connecting to. this works everywhere i have internet access, using my server-side HTTP-to-SSH proxy (example here and here) and proxytunnel (i need to check out corkscrew though).

for those connections and the content delivered from them i have an assortment of Firefox plugins. my current list of Firefox plugins are: Adblock Plus, Cert Viewer Plus, Certificate Patrol, Expiry Canary, facebooksecurelogin, Force-TLS, HTTPS-Everywhere, NoScript, Safe, SSL Blacklist, and WOT.

the end result? i have a lot more insight as to what is going on behind the scenes as i browse the web. every time an old SSL certificate is replaced with a new one, i get a notification with a diff of the changes. when a site's certificate is about to expire i am notified, thus i'll have advance warning if a site could be potentially exploited or unavailable in the future. all connections to frequently-visited sites such as Wikipedia, Facebook, Google, PayPal, Twitter are forced to use SSL. if i connect to an HTTPS page, the border around my browser window is changed to green, certifying the whole page indeed is using SSL. if there is an element in the page which does not use HTTPS, the border is red. if i submit a form at any time and it doesn't go to an HTTPS url, i am warned before i can press submit. if any certificate uses MD5, i am warned. and when browsing google and other websites i am warned if the site has a low or bad rating, has been reported as a malware site, etc (and it's usually right on the money). of course with NoScript any site i don't explicitly trust can't load any potentially-malicious JavaScript, XSS attacks are prevented, and i can even force all cookies and javascript to use SSL to prevent interception or injection (ala Firesheep).

with all these protections i have much more visibility into whether a site i'm on could potentially have malicious content, and my interactions with these and other sites are inherently more secure. of course most of these plugins are only effective on the most popular sites by default, since complex rules often have to be written to allow specific requests to prevent complicated attacks. but at least we're starting to get more secure by default instead of less.

hacking corporate store fronts

local business in america is kind of a quagmire. it seems that except for a few small areas where tons of self-interested stuck-up liberals take the initiative to completely force out corporate interests from a given city, corporations run things. most storefronts you find in america are cost-cutting franchises and subsidiaries of conglomerates. it's no wonder americans easily swallow any pre-packaged product sold to them: be it music, food, television, movies, games... it's all made to order with few variations and dumbed-down for everyone's generic tastes. (heh, it's kind of funny that those are the only things americans are interested in, too)

local businesses get pushed out by these bigger corporations, mostly due to extremely competitive prices. but local business could bring a lot of variety to consumers and in effect influence the entire culture wholly through local means, if it was done on a large enough scale. the question is, in this capitalistic dog-eat-dog country, how do you introduce local business when the whole economy is based on cutting them off at the knees?

i think big companies could start by taking their already extremely effective cost-cutting measures and branching them out to more specific tastes. i think that if you work closer with all the producers of "content" that you use to produce your products you can still keep a low cost and generate a higher variety of products. integrate more, produce with more efficiency.

it would be pretty simple in principle: for any given "metro area" or whatever you determine to be an area with a specific taste that you could market a certain regional product to, create a brand. then create a line of products that are "mostly" only sold by that brand. in this way not only do you create the appearance of originality and variety, you can hopefully win over the local populace and generate a kind of grassroots following for your band.

the goal here is to *not* allow people to associate your stores regionally/locally the way people do nationally. they should not be able to say "that's the mcdonalds of east texas." part of that is keeping your brand relatively small, but also making sure your products aren't overly cookie-cutter in nature. nothing turns people away from big business faster than the lack of a mom-and-pop appearance. you need to hire good people to help sell the brand, but your products also need to have a certain element of being created or finished in the store itself.

have you ever seen a national franchise which could, for example, cook an omlette made-to-order in two minutes for a customer? i don't think i have. there must be an expense associated with shipping fresh eggs, keeping them cool, allowing for a kitchen area to prepare the ingredients, etc. but sandwich/sub franchises do almost this very thing. quiznos franchises receive pre-cooked bread and ingredients and assembles them in a matter of minutes for its customers, and produces what i consider to be a fairly high quality sandwich for the price/time. so why can't we ship pre-mixed eggs and the same ingredients, throw them in a bowl, put it in a microwave or some other omlette-cooking machine and give people something fresh(ish) and made-to-order/home made?

all you'd need to do at that point is rename the store for a given region and customize the ambiance, and switch around the recipe a bit depending on the area. your store fronts gain the reputation of being a "local", original, consistent source for (hopefully) good products, and your customers gain the knowledge that they're not just buying the same old crap from a national chain, maybe even believing they are helping the local economy. (maybe they could even go so far as to put more reward in the hands of the local store owners/managers as to actually produce more good for the given region? but now i'm really dreaming)

Why I think Devops is stupid

http://www.jedi.be/blog/2010/02/12/what-is-this-devops-thing-anyway/

First of all, this isn't a "movement." People have been trying for years to get quality sysadmins who are also competent programmers. I still believe that except for a few rare cases, these people do not exist. And they shouldn't: clearly, something is wrong.

If I told you I spend all of my time both becoming the best sysadmin I can be, and becoming the best programmer I can be, would you believe me? If so, I have a bridge to sell you. The fact is that when i'm a sysadmin I really don't program much at all. I spend my day at work fighting fires and performing odd jobs and when I get home the last thing I want to do is get back to the computer. And at work, if I spent most of my time researching new development trends and writing new tools in experimental languages, how much real sysadmin work am I doing? No, the truth is I wouldn't have enough time in the day to be both a full-time sysadmin and a full-time programmer. I can only do one job at a time.

"the Devops movement is characterized by people with a multidisciplinary skill set - people who are comfortable with infrastructure and configuration, but also happy to roll up their sleeves, write tests, debug, and ship features."

Sorry. I have a job. I don't want to have to do the developers' jobs too. I'm upgrading the Oracle cluster to RAC and being woken up at 3 AM because some bug somewhere deep in the site caused pages to load all funky, and i'm trying to figure out who committed the flaw and get them to revert it. Even if I wanted to, i'm a sysadmin; i'm not familiar with the developers' codebase, and sometimes not even the language they're writing it in. How the hell can you expect me to realistically debug it in real time? And writing tests? Really, you want me to write the developers' unit tests?

Don't get me wrong. I am fully in support of the general idea of better communication between groups and sysadmins working with developers, DBAs, QA, neteng, etc to build a better product. I think it'd be insane for any group to go about making any major changes without consulting every other group and working out any potentially negative ramifications. But this doesn't mean each group has to know how to do each other group's job. Communication is the key word here, not cross-pollination.

There are lots of technical issues that come up in the building of any product. To make it work as well as possible, there's lots of different problems which have to be accounted for. The problems cited in the above post - 'fear of change,' 'risky deployments,' 'it works on my machine,' 'siloization' - all require planning and cooperation to resolve. But this is basic stuff, to me. You don't need to be a DevOp to realize you're going to need your devs to have the same baseline system for testing their apps as your production system (sometimes more than one). The apps have to be developed in a way that allows for a smooth upgrade in the future. And you need a competent deployment and reversion system with change approval/code review and reporting.

These issues are not solved by simply having a 'DevOp', whose responsibility is not only their own systems but apparently the total management and architecting of the whole process of development of a product and delivering it working flawlessly. To properly deal with these issues you need many things. You need really strong management to keep teams working together and to help them communicate. You need some kind of manager or architect position who can keep track of how everything works and juggle the issues before they become serious problems. You need people who are really good at doing their job and get them to ask for help.

Nobody's job is simple. But creating some new position to supposedly solve all these issues by being super-human techno-gods? Even if you could get these godly Devops people in every corporation, there's no promise that they can even get past the politics inherent to each group to make everything work as harmoniously as the post describes. There is no magic bullet. No movement will make everything alright. The world is harsh and complex, and a DevOp isn't going to save it.

utf8 terminals

UTF-8 lovin' for my terminals:

(in bash)
LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
(in irssi)
/set recode_autodetect_utf8 ON
/set term_type utf-8
/set term_charset utf-8
(for your terminal)
uxterm -sb -bg black -fg green -g 100x25
(for screen)
screen -U
(for tmux)
tmux -u

my damn fonts keep having a problem with chinese and other languages if i don't use the default font and size. luckily it's barely usable, but still pretty large. more application-specific details here.

do the legwork

in the various positions in the IT industry we all have a specific job to do with various tasks. we don't always do them as well as we could. usually it boils down to someone doing the bare minimum for a variety of reasons and something ends up breaking.

there are different reasons why things might not be done as well as possible. maybe the deadline's fast approaching and you just need something to work. maybe you've not got enough budget. maybe your bosses are just jerks and even though you tell them what you need to get it done right, they ignore you and force you to produce sub-standard work.

the resulting fail will sit in the background for some time until a random occurrence triggers it. by chance something goes wrong and then everyone breaks, and you're left holding the bag. sometimes that means big hassles and wasted money. sometimes it means you get fired. so when you do have the chance, take the time and do it right.

as far as security is concerned this principle affects everything. there are lots of things you can do to secure any given system. the more you do, the less likely it is that the one attacker you were working to stop will be successful in his or her objective. this applies to everyone in the IT field: programmers, admins, NOC, QA, analysts, managers, etc etc. if you do it all right the first time you won't be left with the bag.

so for example. if you work for a large mobile internet service provider and it's your job to set up the service paywall, don't skimp on anything. make sure it's as secure and reliable as possible and don't trust anything to chance. the one person who figures out that way for everyone in the country to get free internet could bring on considerable strain (financially and otherwise) to your employers, and they won't be happy with you.

or if you run the large systems which are targeted by drive-by botnets as command and control machines or injection points, do your jobs, people. apply the latest security-tightening patches. use mandatory access control. use chroots. use separate users for each service. remove the need to log in as root wherever possible. add intrusion detection. keep up with patches! do you know how much of a hassle it is to clean up and replace systems that have been owned en masse just because you allowed a simple shitty buffer overflow to execute?

and programmers, come on. you're never held responsible for these problems. it's always the other groups which are used as the example and who look foolish because of your crappy, insecure code. the code runs on their systems, so the perception is it's their fault they got owned. but they didn't write that shitty file-uploading php script, you did. you let the bot herders in the front door and made it that much easier for them to expand their attack into the network. congratulations, homie. yes, the admins should have tightened security around php to account for unexpected holes, but you shouldn't make it easier for the attackers either.

and firewall dudes: how hard is it to friggin download a malware watch list and block bad domains/IPs? you're responsible for both the servers AND desktops which are affected by worms/trojans/etc. you know how to tighten these boxes down and tighten up the network access, so do it already!

you're saving yourself work in the end. how many of us have been caught in a tight deadline when suddenly all work has to stop to deal with the intrusion and see how far it got? do you have the spare boxes and cycles to deal with that? how is it affecting your bottom line? your sleep schedule? in the end it's the executives and managers who need to be more proactive in enforcing these trends in the rest of the work force, because if they don't force people to then nobody's going to take the extra time. create a culture of polished work and everyone should benefit.