System Configuration

This document provides some information about setting up your system environment in order to run MIEN. There is also some help about how to use a Unix-style command prompt.

None of the information in this document applies to Microsoft Windows. These techniques work on Linux, Unix, Mac OS X, BSD, BeOS, and any other quasi-posix operating system. Microsoft chooses to do things differently. That's their choice, and I can't stop them form doing it, but they can't make me write documentation for their perverse system configuration methods. You may find some solutions to simple problems on Windows on the Windows-specific page.

Also, most shell examples assume that you use the "bash" shell. This is currently the most common shell, and is the default on Ubuntu Linux and Mac OS X. If you use some other shell, such as "csh", things are a bit different, and some of this documentation may not apply.

Using a terminal

A terminal is a program (or, historically, an entire computer) which provides an environment for running a shell. The shell (also called a command promt, command line, or command line interface). Is program that prints a prompt at which you can enter commands by typing text. The commands are interpreted by the shell or passed on to the operating system. Most operations you can do with a GUI OS are really just shortcuts that call some sort of shell command. For example, clicking on the "Firefox" browser icon in Ubuntu Linux is equivalent to opening a shell, typing "firefox", and pressing enter. The learning curve for using a shell is somewhat steeper than for a GUI, since you have to know what to type. Once you know some commands, though, the shell is often more powerful, more flexible, and faster than the GUI equivalents.

MIEN, and Python, are best used from a terminal. How you open a terminal depends on your OS.

On Macintosh, you can find a terminal program in /Applications/Utilities. The program is named "Terminal" (actually "Terminal.app", but "Terminal" is what will display in the Finder). Double-click on it to open it. I recommend putting a link to Terminal in your dock (unless you like to launch apps with Spotlight or Quicksliver). Run it often. Mac OS 10 is much more powerful than OS <=9, and Terminal.app is your key to harnessing much of that power.

On Ubuntu, you can get a terminal from the main menu with Applications->Accessories->Terminal. On any Linux that provides quick launch commands, try executing xterm or gnome-terminal to get a terminal. Almost all linux flavors that offer a desktop environment with a menu will have a terminal somewhere in the menus. If you don't have a desktop environment, well, you are probably in a terminal already :)

Modern terminals have extensive features for ease of use that rival modern GUI programs. The most useful of these features are tab completion and history.

Tab Completion

If you press the tab key in most shells, whatever you are typing will be completed with an allowed shell command. You may need to press tab twice, if there are more then one possible completions available. This will show you a list of completions. For example, if you are in your home directory, and you have subdirectories "Desktop", "Downloads" and "Documents", then typing cd D[tab] will have no effect. Typing cd D[tab][tab] will print the list "Desktop, Downloads, Documents". You will be returned to your active command prompt. Now you can type an "e" (for a complete entry of "cd De"), and press TAB again. That will complete your command to "cd Desktop". Now you can press enter to execute the command.

Tab completion is context sensitive. If you have already typed "ls" (list files or directories), you can tab complete the names of files and directories. If you have typed "cd" (change to directory) you can only tab complete directories. If you haven't typed any commands yet, you can tab complete the names of executable programs in your system PATH.

Tab completion is great for selecting local file paths to use as arguments. For example, if you have a file "data/10-22-08.0001.anat.dat", and you type mien data/10_22_08.001.anat.dat, you will get an error, because you typed the wrong name. Using "mien da[tab]10[tab]" may work better (unless you have many directories beginning with "da", or many files begining with "10". Then you will need to use more characters to uniquely specify your intended file). In any case, if tab completion provided the file name, you can be sure that it exists, and is spelled right, since tab completion works by reading the directory structure on your computer.

History

If you type a long, complex shell command, and press enter, the command will execute. Sometimes you typed an error, though, or your command worked but you want to do it again. Pressing up-arrow (or CTRL-p) will recall the last command to the current shell prompt, but not execute it. You can edit the command to your satisfaction, and then press enter to execute it again.

The shell history is pretty long. Lots of commands are stored there. If a Unix wizard recently visited your desk and typed a bunch of cool commands, but you don't remember them, your shell probably does.

There are also lots of commands for searching through the history, and for quickly editing a command you pulled from the history. These are beyond the scope of MIEN documentation, but it will save you time to learn a little about the shell and start using its features.

Most people I know who have became facile with shell commands prefers to manage files, configure their system, and run programs from the shell, rather than from the various GUI tools that serve those functions. Also, the GUI tools on different systems are very different (even on different distributions of 2.6 kernel deb-package based Linux!!). The shell is largely the same. There are different shells, to be sure, but if you learn the "bash" shell, you will be able to find it on most modern operating systems (other than Windows), and most of the power and ease of use you learned on one system will still work on another.

Environment Variables

The operating system uses a bunch of variables, known as the environment to figure out where to look for programs and libraries. If you install a program and it doesn't work, a very common cause is that the program itself can't be found in the list of locations to search for programs, or that the libraries the program needs can't be found in the list of locations to search for libraries. Very often you can fix things by setting up the environment variables in your system.

The environment variables most important to MIEN are PATH and PYTHONPATH. PATH tells the system where to find executable programs. The Python interpreter used to run MIEN, and the mien shell command both need to be in directories included in PATH. PYTHONPATH tells the Python interpreter where to look for modules. The mien module needs to be in a location included in PYTHONPATH.

Another commonly used variable is HOME, which is the path to your home directory. MIEN looks for configuration files and extension blocks in your HOME.

You can figure out the value of an environment variable by using the "echo" command to print a value, and preceding the name of the variable with a "$" to get the current value. For example echo $PATH will print out the current value of your PATH variable.

$HOME can also be referred to as "~". The commands cd ~ and cd $HOME are equivalent, and change the current working directory of your shell to be your home directory. My documentation will often use "~" to represent your home directory. For example, if you did a typical subversion install of MIEN, you installed it in the directory "~/mien", which might mean "/Users/gic/mien" if you are me, on a Mac computer, or "/home/bob/mien" of you have user name "bob" and are on an Ubuntu computer. On Unix-like operating systems, the HOME variable is automatically set, and you will never need to change its value (but you may need to know the value, or to put files "in your home directory").

How you set an environment variable depends on your shell. In the bash shell, you would use something like this: export PATH=$PATH:$HOME/bin. The export part of this command tells bash to set the variable for all future shells and programs, not just for the current one. The rest of the command is of the form "X=Y". X can be any variable name, but it will only change the behavior of your programs if it is the name of a recognized environment variable (in our case PATH or PYTHONPATH). In this example, there is a complex string, including several other variable names preceded by "$" in place of "Y". Remember that the "$" tells the shell to evaluate a variable, so assuming your home directory is "/home/bob" and your path is "/usr/bin:/usr/local/bin", then the example above is equivalent to export PATH=/usr/bin:/usr/local/bin:/home/bob/bin. The use of "$HOME" is just a form of shorthand. The use of "$PATH:" allows us to add to an existing PATH, without even knowing what the current one is. This can be rather useful, since the OS, or other installed programs, may have made their own modifications to the PATH, and we don't want to destroy them.

Both PATH and PYTHONYPATH variables specify lists of directories. The ":" character is used to separate paths in the list, so "/usr/bin:/usr/local/bin" means: "The '/usr/bin' directory and the '/usr/local/bin' directory", and "$PATH:$HOME/bin" means "The current PATH, and $HOME/bin".

Now, to set the variables. Suppose you have installed MIEN in $HOME/myprogs, and that you used a compressed archive to install it, so the MIEN package initialization script is now installed at $HOME/myprogs/mien/__init__.py and the mien command is installed at $HOME/myprogs/mien/frontends/mien. Suppose you also installed a custom Python interpreter at $HOME/myprogs/bin/python, and you want to use this Python to run MIEN. Here's what you need to do:

Notice the order in the first line. The $PATH part represents your current PATH. This probably includes some directory like "/usr/bin", and it is reasonably likely that there is a program "/urs/bin/python", which runs some version of Python. The shell looks for programs in the path in the order that directories are listed. By putting $HOME/myprogs/bin before $PATH, you insure that your custom Python gets called. If you put this after $PATH, your python could be found, but it might never run, since the system default python would run instead. The $HOME/myprogs/mien/frontends tells the shell where to find the mien command. It is much less likely that your system has a built in mien command than that is has a built in python, so it is probably safe to list this last.

The second export line tells the Python interpreter where to find the mien package. What you should put here is the name of the directory containing "mien/__init__.py". If you did a developer svn install, and have the file structure: "$HOME/mien/mien/__init__.py", then you should use export PYTHONPATH=$PYTHONPATH:$HOME/mien. If you used a typical end-user svn install, and have "$HOME/mien/__init__.py", use PYTHONPATH=$PYTHONPATH:$HOME, etc.

Now you are set up, but there is one problem left. When you exit this shell, and later start another one, your environment will be back to the default, and MIEN won't work. You need to execute these EXPORT statements every time you open a shell. This is accomplished automatically by adding the statements to your shell run control file. If you use bash, on a normal system, edit (or create) the file $HOME/.bashrc and add the export lines at the end. Here is some more info about run control files.

Shell configuration (run control) files

Unix shells execute several run control files on startup. These files allow the user to execute shell commands every time the shell starts. Editing your local run control files is the normal way to set up your environment variables, aliases, and shell prompt. The operating system provides default run control files, but you can add commands that apply to your personal shell by providing run control files in your home directory. The two run control files are the "profile", and the "rc" file. Different shells assign different names to these files. The most common shell, bash, uses the names ".bash_profile" and ".bashrc". Many other shells use ".profile" and ".[name-of-my-shell]rc". For example, "csh" uses ".profile" ond ".cshrc".

The profile file executes when you open a login shell (this is the first shell opened when you log into your computer). The "rc" file executes every time you open a shell. All of the configurations suggested here can be put in the "rc" file.

The first thing you need to do is find the "rc" file. It will be located in the top level of your home directory, and its name will begin with a ".". The leading "." makes the file invisible to most GUI file managers, and to the default mode of the ls command. You can use ls -a to list files even if they have a leading ".".

Usually when you open a shell, you start out in your home directory. You can make sure you are in your home directory by using the command cd ~.

Now, if your shell is bash, the file you want is named .bashrc. You can use ls -a .bashrc to see if it exists. If it doesn't, you can create it with touch .bashrc. Now, edit it with your favorite editor (eg vim .bashrc) to add some configuration commands.

If you don`t use the bash shell, you may need to edit a different file.

Making aliases

MIEN includes several different GUI applications, which can be used as stand-alone programs, as well as components of the larger MIEN system. The most commonly used programs are CellViewer and DataViewer. Early versions of MIEN provided the commands cv and dv for launching these programs, but current versions provide only the mien command for launching all mien applications. This means that you need to type mien -a cell to launch the CellViewer. If you would rather type cv, you can set this up using the alias command, provided by most shells. Entering the command alias cv="mien -a cell" will cause the shell to replace the cv that you type with mien -a cell when you enter a command. This will allow you to use the cv "command" in the same way you would have used mien -a cell.

The alias command only lasts until you exit the current shell. If you want your new configuration to always work, you need to add the alias command to one of your shell run control files. Find the appropriate file, and include the following lines:

Now you should be able to use the cv and dv commands to launch the appropriate GUIs.

Shell scripts

Shells are more than just input/output interfaces, they are complete interpreted programming languages. When you type "ls" at a bash prompt, you are essentially running a tiny 1 line bash program. It is possible to write more elaborate shell programs as well, and you can save them in files an run them as scripts, or run them directly from the shell prompt. In fact, a large part of the normal operation of a modern Linux operating system is controlled by the execution of shell scripts. In the early 90s, several respected computer scientists suggested that the Unix shell language (the bash shell in particular) was "the most advanced programming language extant". We certainly have better languages now (like Python :), but the fact remains that the shell is a complete, and powerful, programming language, and this means you can use it to write useful programs (called shell scripts).

Since MIEN is usually called from the shell, the mien command is easy to use in shell scripts. The topic of shell scripting is beyond the scope of this documentation, but I will present a useful example that may 1) Solve a frequently asked question about MIEN usage, and 2) whet your appetite for learning more shell-fu.

Suppose you have a bunch of ".swc" files in your current working directory, and you want to share them with a collaborator who only has a viewer for Neurolucida (asc) files. Since you have just installed mien, you are hoping that it can convert your files to the asc format. You are in luck, it can! You read some documentation, and concluded that mien -c asc file001.swc will produce a file named "file001.asc" in the right format. The only problem is that you have 200 of these files. You could, perhaps, type 200 mien -c commands? No. Don't do it. The whole point of a computer is that it does stupid repetitive tasks for you so that you don't have to. Any time that you do the same operation 5 times in a row, on your computer, you can be sure that there is a better way (unless, of, course, you are playing Halflife 2, and you enjoy shooting zombies over and over).

The obvious thing to do is use a shell glob. ls *.swc will list all of your swc files, so mien -c asc *.swc should convert all of them. Unfortunately, this doesn't work as you want it to. MIEN reads all the swc files, and concatenates them into one file, which it converts to a single asc file. Drat. You could complain to me, and ask me to change the behavior of mien -c, or, if you write Python, you could change it yourself (If you write Python, you might also build a batch conversion tool in Python, in about 4 lines of code, but the shell script I'm going to show you is even easier). The thing is, concatenation is often useful. When you type mien -c mien *.swc, you probably want concatenation, and when you type mien -a cell *.swc you certainly do. How, then, do we get around this (very common) problem of converting files to asc? The simplest answer is an interactive shell script. Use this command:

This operates as a single shell command, and it does what you want: it converts the 200 swc files to 200 separate asc files.

If you know some programming, you can already see that this is not just a command, it is a program. In particular it is an example of the common control structure called a "for loop". The ";" characters separate lines (as they do in C, perl, and other languages), so there are really three instuctions; the "for" line, which specifies a list of things we are going to iterate over, the "do" line, which says what to do with each thing in the list, and the "done" line, which says we are done with the loop, and don't need to do anything else with each thing. The "*.swc" is a shell glob, and acts the same way here as it would in any other context: it is equivalent to a list of all the file names matching the pattern "*.asc". In this case that is a list of 200 swc files. The string "fname" is a variable, and it gets, iteratively, bound to every one of those 200 names. The "do" line thus gets done 200 times, with "fname" carrying 200 different values. Putting "$" before a variable in shell retrieves its value, so the shell command specified in the "do" line is mien -c asc [CurrentValueOfFname], and it gets called 200 times, once for each possible value of fname.