This is a braindump of an idea. I want a minimal, simple 'tool' to manage my configuration across multiple hosts. I have configs I share between work and home and my colo box, all with minor differences. Cfengine and Puppet take too much time for me to set them up properly, so i'll just write something which will take slightly longer but be simpler in the end. added Above all, this tool should not make silly assumptions for you about how you want to use it or what it should do - it should just do what you tell it to without needing to know finicky magic or special parameters/syntax.
I already have subversion set up so i'll stick with that, otherwise i'd use cvs. I need the following functionality:
* auto-update sources (run 'svn up' for me)
* check if destination is up to date, and if not, apply configuration
* search-and-replace content in files
* try to maintain state of system services
* send an alert if an error occurs
* added run an external command to perform some action or edit a file post-delivery
That's about it for now. Mostly I just want it to copy files based on the host "class" (colo, home, laptop, work). Since I want it to run repeatedly the alert is only because it'll be backgrounded; otherwise it will obviously report an error on stderr. It should also be able to run on cmdline, background, etc and lock itself appropriately. Logging all actions taken will be crucial to making sure shit is working and debugging errors.
I don't want it to turn into a full-fledged system monitoring agent. The maintain state of system services is more of a "check if X service is enabled, if not enable it" thing, not running sendmail if sendmail isn't running. On Slack this is about as complicated as "is the rc.d file there? is it executable?" but on systems like Red Hat it's more like "does chkconfig show this as enabled for my runlevel?". I don't know what tool i'll use to monitor services; I need one but it isn't in scope of this project.
Now for a name... it's going to be checking that my configuration is sane, so let's go with "checkup". There doesn't seem to be an open source name conflict.
For config files i'm usually pretty easygoing. Lately i've been digging on .ini files so i'll continue the trend. To make the code more flexible to change i'll make sections match subroutines, so I can just make a new .ini section and subroutine any time I wanna expand functionality. Syntax will be straightforward and plain-english, no strange punctuation unless it makes it more readable. Multiple files will be supported, though if a file fails syntax check by default it'll be ignored and the rest of the files will be scanned and a warning thrown. If any file references anything which is missing an error will be thrown and exit status non-zero. Each section will also be free-form: the contents of the section determines what it's doing. If it defines hostnames and a hostgroup name, the name of the section should probably implicitly be a hostgroup class name. A list of files and their permissions, etc would mean that section name imposes those restrictions on those files. A set of simple logic conditionals will determine if that class can be evaluated (ex. "logic = if $host == 'peteslaptop'"; there is no "then blah blah" because this is just a conditional to be evaluated, and if it's false that class is ignored). added The conditionals will evaluate left-to-right and include 'and' and 'or' chaining (i don't know what it's actually called in a language). While we're at it, a regex is probably acceptable to include here ('=~' in addition to '=='). Hey, it is written in Perl :)
added The tool should be able to run by a normal user as any other unix tool is. It should be able to be fed configs via stdin for example. It will not serve files as a daemon as that is completely out of scope of the tool; in fact, it shouldn't really do any network operations at all save for tasks performed as part of the configs. In addition, all operations should be performed locally without the thought or need to retrieve files or other information - all updates should happen first before any configs are parsed. If for some reason that poses a scalability or other problem we may allow configs to be parsed, then files copied to local disk, then examining the system to execute the configs etc.
The only thing i'm not really sure about is roll-back. I always want roll-back, but it's hard to figure out how to perform such a thing when you've basically just got a lot of rules that say how stuff should be configured at a glance - not how it should look in X time or whatever. Rolling back a change to the tool's configs does not necessarily mean your system will end up that way, unless you wrote your tool's configs explicitly so they always overwrite the current setting. For example, you might have a command that edits a file in place - how will you be able to verify that edit is correct in the future and how would you be able to go back to what it was before you made your edit?
Probably the easiest way to get at this would be to back up everything - make a record of exactly the state of something before you change it and make a backup copy. stat() it, then if it's a file or directory mark that you'll back it up before doing a write operation. Really the whole system should determine exactly what it's going to do before it does it instead of executing things as it goes down the parsed config.
Let's say you've got your configs and you're running an update. One rule says to create file A and make sure it's empty. The next rule says to copy something into file B. The next rule says to copy file B into file A. Well wait a minute - we already did an operation on file A. What the fuck is wrong with you? Was this overwrite intentional or not? Why have the rule to create file A if you were just going to copy file B on top of it? This is where the 'safety' modes come in. In 'paranoid' mode, any operation such as this results in either a warning or an error (you should be able to set it to one or the other globally). Alternatively there will be a 'trusting' mode wherein anything that happens is assumed to happen for a reason. For operations in 'trusting' mode, if there is a rule which explicitly conflicts with another - you told it to set permissions to 0644 in one rule and 0755 in another rule - a warning will be emitted, but not nearly as many as in 'paranoid warning' mode. Of course all of this will be separate from the inherent logging in any test or operation which is performed for debugging purposes.
All this is a bit grandiose already for what I need, but might as well put it in now than have to add it later.
added The config's sections have the ability to be "include"'d into other sections to provide defaults or a set of instructions where they are included. In this way we can modularize or reuse sections. In order to make 'paranoid mode' above happy we may also need to add a set of commands to perform basic I/O options. Also we should be able to define files or directories which should be explicitly backed-up before an operation (such as a mysterious 'exec' call) might modify them. Of course different variables and data types may be necessary. Besides the global '$VARIABLE' data type, we may need to provide lists or arrays of data and be able to reproduce it within a section. '@ARRAY' is one obvious choice, though for data which may apply only to a given section at evaluation-time (such as an array that changes each time it is evaluated) we may need a more specific way to specify that data type and its values.
added The initial configuration layout has changed to a sort of filesystem overlay. The idea is to lay out your configs in the same place as they'd be on the filesystem of your target host(s) and place checkup configs in directories which will determine how files will be delivered. I started out with a puppet-like breakout of configs and templates, modularizing everything etc. But it quickly became tedious to figure out the best way to organize everything. Putting it all in the place you'd expect it on disk is the simplest. You want an apache config? Go edit checkup/filesystem/etc/apache/ files. You want to set up a user's home directory configs? Go to checkup/filesystem/home/user/. Just edit a .checkuprc in the directory you want and populate files as they'd be on the filesystem (your checkuprc will have to reference them for them to be copied over, though).
No comments:
Post a Comment