Consolidating development environments – a Bash Magic tutorial

Developers have a tendency to not only work on a single project at once.  Depending on those projects, there is a constant struggle to keep your programming environment in sync with what you are actually doing.  For that big legacy product you are maintaining you might need an old Java 1.5 in a specific version – for that fancy new web-app you might be using the newest Java.
Different people have different strategies on how to take care of that problem.  IDEs can be used to set software versions for specific projects, but using your build-tool from the commandline uses a completely different environment.  Adapting your current command line environment is a little trickier and can solved in various ways – with varying capabilities and problems.
This article is aimed at anyone wanting to learn a little more about Bash background magic – the resulting code will be workable but will have to be fleshed out by the user to his own needs.
So, what do we actually want our “system” to do:

  • Control build environment (Java, Maven, …)
  • Special parameters for the build/execution environment (MAVEN_OPTS, …)
  • Switching environments while working
  • Switching environments based on project
  • Different environments in different terminals
  • More than one user should be able to reuse “environment profiles”

The first idea on how to switch an environment seems to be to create symlinks to the correct environment.  That can be partly automated but isn’t really pretty and the environment is set for all terminals.
The way I will present here isn’t new, in fact http://rvm.io/ probably does something very similar for Ruby (and is a lot more fleshed out most likely).  But the technique is quite nice and makes switching environments quite unproblematic.
Having multiple users use the same kind of environment of course takes a little standardizing.  The software for the build environment should be located at the same place for example. Our DevOps use Puppet to automate that, but that kind of automation isn’t really a requirement to use anything in this post.
Lets have a look how the heart of the technique looks like:

$ foo() { echo foo; }
$ PROMPT_COMMAND=foo
foo
$

Bash lets you run a command before each prompt.  The command is able to influence the current environment – this is important, as you want to set the profile for the current shell and don’t want to spawn subshells all the time.  It can also set how the next prompt will look like, using the PS1 environment variable.
Lets take that idea, put it in a file and build a little bit of code around it to build a quite basic profile switcher (which we will call profilehandler or ph for short).

profilehandler() {
  if [ x"$PH_PROFILE" = x"new" ]; then
    export JAVA_HOME=/opt/software/jdk/jdk1.7.0_04
  elif [ x"$PH_PROFILE" = x"old" ]; then
    export JAVA_HOME=/opt/software/jdk/jdk1.5.0_22
  fi
  if [ -n "$PH_PROFILE" ]; then
    export PATH="$JAVA_HOME/bin:$PATH"
  fi
  export PS1="${PH_PROFILE:+[$PH_PROFILE] }\$ "
}
PROMPT_COMMAND=profilehandler
$ source profilehandler.sh
$ PH_PROFILE=new
[new] $ echo $JAVA_HOME/bin # -> 1.7

Good:

  • Can set profile
  • Shows profile in path

Bad:

  • Ugly interface
  • Leaks paths into PATH
  • Profiles are specified directly in code

Lets refactor that a little and make it slightly easier and safer to use.

# set up where our software/profiles will reside
PH_HOME=$HOME/tmp/ph
profilehandler() {
  local profilepath
  # check if we have a valid profile
  if [ -f "$PH_HOME/profiles/$PH_PROFILE" ]; then
    profilepath="$PH_HOME/profiles/$PH_PROFILE"
  fi
  # clean up
  local cleanpath="$(echo "$PATH" | sed 's#\('$PH_HOME'[^:]*:\)\|\(::\)##g')"
  export PATH="$cleanpath"
  # a profile can be loaded, do it
  if [ -n "$profilepath" ]; then
    . "$profilepath"
    export PATH="${JAVA_HOME:+$JAVA_HOME/bin:}$PATH"
  fi
  # set prompt
  export PS1="${PH_PROFILE:+[$PH_PROFILE] }\$ "
}

Let the pain of that code dump subside a little bit, then proceed reading.  We don’t do much more than the previous snippet, just load the profile from an external file (in $PH_HOME/profiles) and make sure that the PATH gets cleaned up.  This method of clearing is quite fickle – you need to have all software in PH_HOME and certain edge cases do not get handled – but for our simple requirements, that’s just fine.
Lets also create a nice UI so we can set our profile more elegantly.

# make a nice useable interface
ph() {
  case "$1" in
    set)
      if ! [ -f "$PH_HOME/profiles/$2" ]; then
        echo >&2 "Bad profile: $2"
        return
      fi
      export PH_PROFILE="$2"
      ;;
    enable)
      ph set "${2:-default}"
      export oldPS1="$PS1"
      export PROMPT_COMMAND=profilehandler
      ;;
    disable)
      unset PH_PROFILE
      unset PROMPT_COMMAND
      export PS1="$oldPS1"
      ;;
    *)
      echo "Usage: ph enable [profile]|disable|set profile"
  esac
}

We now have some sanity checking, a way to enable and disable our environment switcher and some amount of cleanup code so we get our old prompt back.
Good:

  • Can set profile
  • Shows profile in path
  • Nicer interface
  • Profiles easily addable

Bad:

  • Insufficient cleanup (PATH retains Java path on disable)
  • Lacks features

But hey, that’s only our second take.  And there is much more room for improvement.

  • Automatic profile switching on project directories (maybe implemented by a dotfile in the project directory – the profilehandler can search down the stack for that and set the profile accordingly)
  • Local and global profile paths, so shared profiles can be distributed easily and users can override global settings
  • Configuring how the prompt looks
  • Showing the branch of your favorite SCM inside the prompt
  • Allow profiles to do their own cleanup/initialize
  • Support more than just Java
  • List available profiles
  • Much more exhaustive environment cleanup on disable

Having the capability to call a function before each prompt and in the scope of the current shell is very handy.  But that also means you have a limitation on how much you can do.  Adding features is a good thing, but you have to take care to not extend the running time to a user-noticeable amount.  You should make sure to only call programs which are sure to return in a short amount of time.  If you are working partly on a network mounted disk, keep in mind that this could potentially block your shell for an extended amount of time.  If you’re using automount and something (like looking up the current GIT branch) in your profilehandler is looking at each directory up to the root, this also can hurt performance.  Also, while extending the profilehandler, if you get a stuck shell which does not show a prompt anymore, keep an eye open for programs reading from stdin.
No matter which method you are using, it is probably a good idea to document your environment settings so everyone in the project has a similar setup.  And automating that process also sounds like a very good idea.  So have fun playing around!