Understanding Environment Variables and the Unix Path
When I was new to Mac and Linux, environment variables and the Unix
PATH were mysterious and confusing. Documentation for various tools would frequently make references to "add this to your path" or "set this environment variable" and I had no idea what they meant.
Fortunately, this seemingly-magic behavior is relatively transparent once you know where to look and understand some of the basic assumptions in Unix. By the time you're done reading you'll be on your way to mastering environment variables.
ENV In 5 Minutes or Less
When you type the name of a program
grep, your shell looks in all the directories specified in your
PATH to try to find a matching program.
PATH itself is an environment variable (other common ones include
JAVA_HOME). Environment variables are global variables that live in your shell session, and help your shell fill in shortcuts or specify preferences.
You can dynamically set (or change) environment variables using
export, or persistently set the values in your
~/.bashrc file. For example, we set
vim, then use
$EDITOR to invoke
vim on our
.bash_profile so we can persist some other environment variables:
$ export EDITOR=vim $ $EDITOR ~/.bash_profile export PATH=$PATH:/something/i/need/to/add export NEW_ENVIRONMENT_VARIABLE=value
Environment variables are read from
.bash_profile at the start of the shell session, so after you save the file you'll need open a new terminal window before the changes will take effect.
If the extremely terse example above doesn't satisfy you, keep reading. (Also, if you got stuck in
:quit! and press
ENTER to exit.)
A Primer on
bash is the standard shell on OS X and most Linux distros.
bash is both an interactive shell and a scripting language. Examples of the latter case are called "bash scripts".
~/.bash_profile is actually a bash script which, by convention, is executed every time bash starts up (i.e. whenever you open a new terminal). So before we go further, let's get familiar with a bit of bash syntax.
bash Prompt Conventions
$ followed by a space is used in documentation to indicate the
bash prompt, so you will frequently see this indicating that you should type something at the terminal. (Don't type the
$, obviously.) E.g:
$ export EDITOR=vim
When you see a line in this context that does not start with
$ it generally refers to output from the preceding command.
exported variable names should be
UPPERCASED, or Bad Things will happen.
echoprints the value of any variable to the terminal.
- Variables are assigned using
VARIABLE=and referenced using
$VARIABLE. Spaces are not allowed between the variable name and the equals sign.
exportallows us to set a variable in our current environment (typically a shell session), otherwise the variable dies at the end of the current script.
You can play around with these on your terminal:
$ FOO=cake $ echo $FOO cake
In bash alone, this has essentially the same effect:
$ export BAR=pie $ echo $BAR pie
If you want
BAR to become an environment variable and be available to other programs you must use
Notice that you can't add spaces around the
= when doing variable assignment:
$ BAR = pie BAR: command not found
Also, if you don't use the dollar sign when referencing a variable you'll just output a string:
$ echo $BAR pie $ echo BAR BAR
Finally, let's put it all together to modify an existing variable:
$ FOO="I like $FOO$BAR" $ echo $FOO I like cakepie
Yes, cakepie is delicious. Anyhow, this is the basis of how we'll modify our
PATH environment variable. (Because someone smarter than us put a lot of useful things in there, and if we override
PATH instead of appending things we'll break our shell. Which is a Bad Thing.)
If you want to learn more about
export, etc., type
man bash at your terminal to read the man page, or read it online.
As in the examples above, an environment variable is simply a variable that has been
exported into an environment (usually your shell session) and can be referenced by other programs.
export EDITOR=vim export PATH=/home/cbednarski/pythons/27/bin:/usr/local/bin:/usr/bin:/bin
exported from the terminal:
$ export EDITOR=vim
If you have
php installed on your system you can run this:
$ php -r "echo getenv('EDITOR');" vim
Or if you have
$ python -c "import os; print os.environ['EDITOR']" vim
Which demonstrates that the
EDITOR environment variable we
exported is now available to
python during runtime. This is a common pattern for configuring command-line tools, since the environment variable is easy to change in terminal session or on a user-by-user basis across a single system.
PATH to Success
PATH is a special environment variable. The "full path" to a file represents its exact location on the filesystem, such as
/usr/bin/bash. Any time you type the name of a program without a full path,
bash searches your
PATH to find it. Here's a simple example of my
PATH is searched from left to right, with
: as a separator. So when I type
python in my terminal, bash will look in the following places (in order) to find it:
If it finds an executable named
python, it will run that program. If bash doesn't find it, we'll see
python: command not found.
Wait! What if bash finds more than one
It won't. As soon as bash finds
python in one of those paths, it executes it and stops searching.
Which Program did
If you have multiple pythons in your path you can use
which python to see which one is being executed, or
which -a python to enumerate all of them. The left-to-right precedence in
PATH is a key feature, since it allows you to override system defaults.
which is a great way to see whether your path is configured correctly.
The safest way to add something to your path is to append it. Note that you do not name the executable itself in the path, just the folder that it sits in.
If you want to override the system default for a program, you should prepend instead:
If you want to make persistent modifications to
PATH, you'll need to make these changes in
~/.bash_profile on your computer. If you just want to mess around and throw away the changes later you can run these commands in your terminal (as we did earlier in the "Primer on bash" section).
Also, remember that you'll need to open a new terminal before changes made to
~/.bash_profile take effect.
Stepping into the Path
Sometimes it's more convenient to place something in the
PATH instead of changing the
PATH itself. You can accomplish this easily using symlinks. The format is:
$ ln -s TARGET ALIAS
/usr/local/bin is the conventional place to make customizations like this, so I could do this and have many versions of python on my system:
$ ln -s /home/cbednarski/pythons/27/bin/python /usr/local/bin/python27 $ ln -s /home/cbednarski/pythons/33/bin/python /usr/local/bin/python33
Then I invoke them using:
$ python27 $ python33
Normally, bash would only find one of these programs because they're all named
python, but with the symlink trick I can make their names unique and still run them easily without having to type in the full path each time.
Appendix A: The Executable Bit
Unix file permissions include a special flag that indicates whether or not you're allowed to run a script as an executable. If you placed something in your path but you can't run it by typing the name, make sure the executable bit is set:
$ chmod +x /home/cbednarski/pythons/27/bin/python
You can verify this by running
ls -l and looking for
x in the file permissions:
$ ls -l /home/cbednarski/pythons/27/bin/python -rwxrwxr-x 1 cbednarski cbednarski 2989480 Dec 16 17:28 /home/cbednarski/pythons/27/bin/python
You may need elevated permissions (e.g.
sudo) to change this.
Appendix B: The shebang, or "Which program do I use to run this file?"
You may have noticed in some files a pattern that looks like this:
This is called a "shebang" or "hashbang". When you write an executable script you need to instruct bash where to find the interpreter to run it. For example, bash doesn't understand python, and the python interpreter can't run php code, so if you want to write a script in python or php and make it executable, you'll need the proper shebang line.
The most common form of the shebang,
#!/usr/bin/python, is actually not correct. You might observe that it doesn't look in our
PATH, and instead is hard-coded to look in a particular place on the file system. What if system python on our computer is version 2.6, but we want to use 2.7? Or what if
python doesn't live in
For optimum portability, the best way to write this is:
/usr/bin/env bit itself is still a hard-coded convention, but it's more flexible since it allows us to use
PATH expansion to find which
python interpreter to use without needing to modify the original script.