Difference between revisions of "Virtual Machines"

From WormBaseWiki
Jump to navigationJump to search
 
(6 intermediate revisions by 2 users not shown)
Line 72: Line 72:
 
=== Navigating WormBase ===
 
=== Navigating WormBase ===
  
After logging in, a browser window will appear taking you directly to your local copy of WormBase.
+
After logging in, a browser window will appear taking you directly to your local copy of WormBase. If it does not, open Firefox and point it to:
 +
 
 +
http://localhost/
  
 
=== Advanced uses ===
 
=== Advanced uses ===
Line 102: Line 104:
 
  > ~/wormbase/bin/pull_software.sh
 
  > ~/wormbase/bin/pull_software.sh
  
Note that you still need to download the databases for each new release!
+
Note that you still need to download the databases for each new release! Until this is scripted, the general process looks like:
 +
 
 +
1. Shutdown the VM
 +
2. cd /your/vm/path/WS181
 +
3. Remove symlink: rm -f current_databases
 +
4. Fetch new databases:  curl -O ftp://ftp.wormbase.org/pub/wormbase/people/tharris/vmx/WS182/WS182-databases
 +
5. Create symlink: ln -s WS182-databases current_databases
 +
6. Unpack databases: cd WS182-databases ; tar xzf *.tgz
  
 
=== Mailing List ===
 
=== Mailing List ===
Line 122: Line 131:
 
2. When launching the virtual machine, for networking, select "Bridged".
 
2. When launching the virtual machine, for networking, select "Bridged".
  
3. Configure network settings with your IP, domain name, and subnet mask.
+
3. Configure network settings with your IP, domain name, and subnet mask.
  
 
In this example, the guest OS IP is 143.48.220.208. This should be changed to whatever your assigned IP address is.
 
In this example, the guest OS IP is 143.48.220.208. This should be changed to whatever your assigned IP address is.
Line 128: Line 137:
 
ifconfig eth0:0 143.48.220.208 netmask 255.255.255.0 broadcast 143.48.220.255 route add -host 143.48.220.208 dev eth0
 
ifconfig eth0:0 143.48.220.208 netmask 255.255.255.0 broadcast 143.48.220.255 route add -host 143.48.220.208 dev eth0
  
You can also do this from the GUI if you prefer, under System Settings -> Network. Double click on the network adaptor.
+
You can also do this from the GUI if you prefer, under System Settings -> Network. Double click on the network adaptor. To use the GUI network setup tools you need to be logged in as root.  The root login/password is root/wermbace. 
  
 
  Address:  Your assigned IP address
 
  Address:  Your assigned IP address
Line 180: Line 189:
  
 
By default, WormBase virtual machines use NAT networking. If you would like to use a server in NAT mode, you will need to configure an upstream proxy server (Apache or Squid, for example) to access the virtual machine remotely.
 
By default, WormBase virtual machines use NAT networking. If you would like to use a server in NAT mode, you will need to configure an upstream proxy server (Apache or Squid, for example) to access the virtual machine remotely.
 +
 +
First, configure the network on the virtual machine. 
 +
 +
1. From within the VM login as root.  The root login/password is root/wermbace.  To use the GUI network setup tools you need to be logged in as root.
 +
 +
2. Configure the NAT networking.  Set the MAC address on the VM.  Go to the menu 'Network configuration'->'Devices' tab->'Edit' (eth0)->'Hardware Device'->click on 'Probe' to get the correct MAC address.  Then click on 'Activate' for eth0.
 +
 +
3. Find my NAT IP address in the virtual machine.  Open a terminal (right click on the Desktop, select 'Terminal') and enter:
 +
/sbin/ifconfig
 +
Look in the eth0 section for the 'inet addr:' line.
 +
 +
Now configure proxying.  Here is a simple setup for a host machine with two hostnames and one IP address running the Apache webserver.
 +
 +
Edit httpd.conf (typically located at /etc/httpd/conf/httpd.conf).  The example below sets wormbase.univ.edu as the wormbase VM hostname and myserver.univ.edu as the hostname for the host server.
 +
 +
ProxyRequests Off
 +
<Proxy *>
 +
    Order allow,deny
 +
    Allow from all
 +
</Proxy>
 +
 +
NameVirtualHost *:80
 +
 +
<VirtualHost *:80>
 +
ServerName myserver.univ.edu
 +
ServerAlias *.myserver.univ.edu
 +
UseCanonicalName off
 +
DocumentRoot "/var/www/html"
 +
</VirtualHost>
 +
 +
<VirtualHost *:80>
 +
ServerName wormbase.univ.edu
 +
ServerAlias *.wormbase.univ.edu
 +
ProxyPass / http://172.16.236.128/
 +
ProxyPassReverse / http://172.16.236.128/
 +
</VirtualHost>
 +
 +
In the example above, 172.16.236.128 was the IP of the virtual machine.  Restart the webserver and the proxying should take effect.
  
 
= Troubleshooting =
 
= Troubleshooting =
Line 192: Line 239:
  
 
Double click on the network adaptor and select the "Hardware" tab. Click on "Probe", then "OK".
 
Double click on the network adaptor and select the "Hardware" tab. Click on "Probe", then "OK".
 +
 +
 +
 +
[[Category:User Guide]]
 +
[[Category:Developer documentation]]

Latest revision as of 18:43, 17 August 2010

Overview

"Virtual Machines" (or VMX) are self-contained packages of everything you need to use WormBase locally

Virtual machines can be "played" on Mac, Windows, and Linux platforms using the free VMPlayer (Linux, Windows) or the (non-free how do you like them apples?) VMWare Fusion Player (Mac) available from VMWare.

To keep downloads as reasonable as possible, we've split virtual machines into two components. First, you will need the primary virtual machine, which contains the WormBase software, within a Linux operating system.

Once you have the software, you can download the datasets you are interested in. We supply these as "virtual disks" -- consider these removable disks exactly like you might have sitting next to your laptop. Download only what you need, such as C. elegans predictions, WormMart, etc, although to keep things simple, we recommend you download everything.

Running a virtual machine of WormBase is incredibly easy.

  • Download the VMPlayer from VMWare
  • Download a WormBase virtual machine
  • Select the datasets you'd like to use and download them
  • Open the virtual machine using VMPlayer or VMFusion
  • Log in
  • A browser launches automatically and takes you to your copy of WormBase!

System Requirements

Available Virtual Machines

WormBase virtual machines are preconfigured for use on the desktop. This means that they are intended to be run and used on the same local machine. If you would like to run the virtual machine as a server you will need to make some small changes to the configuration. See the section "Running a WormBase Virtual Machine as a server" for additional details.

Please see the WormBase FTP site for available virtual machines.

Download Instructions

The easiest way to fetch a virtual machine is through use of the command line.

1. Open a terminal window

2. Try using curl or wget:

curl -O ftp://ftp.wormbase.org/pub/wormbase/people/tharris/vmx/WSXXX

Once you've completely downloaded one of the releases, unpack each of the components using tar/gzip.

The directory structure should look like this, where "current_databases" is a symlink to WSXXX-databases.

WSXXX/
    |
    --wormbase-live-server/wormbase.vmx
    --WSXXX-databases/
      |
       --acedb/
       --autocomplete/
       --c_elegans/
       --other_species/
       --support/
    --current_databases -> databases

Note: Do make sure that you have 30-50 GB of disk space free prior to unpacking!

Using a WormBase Virtual Machine

Starting the Virtual Machine

Launch your Virtual Player / VM Fusion. From the File menu, select "Open". Navigate to the unpacked directory of the WormBase Virtual Machine. Open it. Inside, you will find a file ending with the suffix ".vmx". Select it, then select OK. The Virtual Machine will start to boot up.

Logging in

User: wormbase
Pass: wormbase

Navigating WormBase

After logging in, a browser window will appear taking you directly to your local copy of WormBase. If it does not, open Firefox and point it to:

http://localhost/

Advanced uses

Each WormBase Virtual Machine contains the entirety of WormBase. This makes it a great tool for tasks like data mining. You can also use individual components of WormBase to best suit your needs.

Xace - the graphical interface to AceDB

 /usr/local/acedb/bin/xace /usr/local/acedb/elegans

Tace - the text interface to AceDB

 /usr/local/acedb/bin/tace /usr/local/acedb/elegans

AcePerl - Perl API to AceDB

Mine the underlying AceDB database programmatically using AcePerl.

Bio::DB::GFF - Perl API to genomic annotations

Each virtual machine contains multiple mysql databases of genomic annotations, including (as of WS170, 2/2007), C. elegans, C. briggsae, and C. remanei. You can use the Bio::DB::GFF API -- part of BioPerl -- to easily mine these sequence annotations.

Staying Up-To-Date

Updating the Software

It isn't necessary to download a new Virtual Machine for every release of the database. To keep your virtual machine up-to-date, just leave it running overnight. It will automatically fetch new software every evening. If you would like to do this manually, open a Terminal and run the following command.

> ~/wormbase/bin/pull_software.sh

Note that you still need to download the databases for each new release! Until this is scripted, the general process looks like:

1. Shutdown the VM
2. cd /your/vm/path/WS181
3. Remove symlink: rm -f current_databases
4. Fetch new databases:  curl -O ftp://ftp.wormbase.org/pub/wormbase/people/tharris/vmx/WS182/WS182-databases
5. Create symlink: ln -s WS182-databases current_databases
6. Unpack databases: cd WS182-databases ; tar xzf *.tgz

Mailing List

I've set up a very low traffic mailing list for users interested in running WormBase locally. Please subscribe. I promise I won't spam you.

Running a WormBase Virtual Machine as a server

Each WormBase Virtual Machine contains everything that you need to run it as a standalone server. In server mode, you can install a single copy of WormBase locally and make it available to others in your lab or organization, or even establish your own local mirror of WormBase accessible to all.

To do this, you will need to modify a few settings of the virtual machine. The main consideration for running a virtual machine as a server is how to manage networking. Described below are the two most suitable approaches. Bridged networking is the easiest and requires no additional components.

Using a VM with bridged networking

When run in bridged networking mode, the VMX shares the host OSs internet connection. This allows the guest OS to run with its own domain name and IP address.

1. Acquire a static IP and suitable domain name for your virtual machine

2. When launching the virtual machine, for networking, select "Bridged".

3. Configure network settings with your IP, domain name, and subnet mask.

In this example, the guest OS IP is 143.48.220.208. This should be changed to whatever your assigned IP address is.

ifconfig eth0:0 143.48.220.208 netmask 255.255.255.0 broadcast 143.48.220.255 route add -host 143.48.220.208 dev eth0

You can also do this from the GUI if you prefer, under System Settings -> Network. Double click on the network adaptor. To use the GUI network setup tools you need to be logged in as root. The root login/password is root/wermbace.

Address:  Your assigned IP address
Subnet mask: 255.255.255.0
Default gateway: 143.48.220.254
Broadcast host:  143.48.220.255 (not explicitly set in the GUI)

4. Reset the MAC ID of the guest

System Tools > Network

Double click on the network adaptor and select the "Hardware" tab. Click on "Probe", then "OK"

5. Add the following lines to /etc/resolve.conf for DNS. For our example, these are:

search cshl.edu
nameserver 143.48.1.1
nameserver 143.48.1.20

6. Set the hostname

This can be done either in the GUI under the Network panel, or using the following command line terms.

If you have a static IP address, then /etc/hosts is configured as follows:

127.0.0.1	           localhost.localdomain      localhost
143.488.220.44 mybox.mydomain.com	mybox

After updating the /etc/hosts file correctly, the "hostname" command should be run as follows to set your hostname:

hostname mybox.mydomain.com

7. Edit /usr/local/wormbase/conf/localdefs.pm and httpd.conf with the appropriate hostname as necessary.

8. Place the VM into console mode:

Comment out the runlevel 5 directive and uncomment the id3 runlevel directive in /etc/inittab:

$ sudo perl -p -i -e 's/id:5:initdefault:/#id:5:initdefault:/' /etc/inittab
$ sudo perl -p -i -e 's/#id:3:initdefault:/id:5:initdefault:/' /etc/inittab

9. Shutdown the virtual machine and copy it as a backup

I append "server" to the name to indicate that it is configured as a server

  tar czf wormbase-WS100.2003.05.13-server.tgz wormbase-WS100.2003.05.13

10. Restart the Virtual Machine. You should be good to go!

Using a VM with NAT addressing

By default, WormBase virtual machines use NAT networking. If you would like to use a server in NAT mode, you will need to configure an upstream proxy server (Apache or Squid, for example) to access the virtual machine remotely.

First, configure the network on the virtual machine.

1. From within the VM login as root. The root login/password is root/wermbace. To use the GUI network setup tools you need to be logged in as root.

2. Configure the NAT networking. Set the MAC address on the VM. Go to the menu 'Network configuration'->'Devices' tab->'Edit' (eth0)->'Hardware Device'->click on 'Probe' to get the correct MAC address. Then click on 'Activate' for eth0.

3. Find my NAT IP address in the virtual machine. Open a terminal (right click on the Desktop, select 'Terminal') and enter:

/sbin/ifconfig

Look in the eth0 section for the 'inet addr:' line.

Now configure proxying. Here is a simple setup for a host machine with two hostnames and one IP address running the Apache webserver.

Edit httpd.conf (typically located at /etc/httpd/conf/httpd.conf). The example below sets wormbase.univ.edu as the wormbase VM hostname and myserver.univ.edu as the hostname for the host server.

ProxyRequests Off
<Proxy *>
   Order allow,deny
   Allow from all
</Proxy>
NameVirtualHost *:80
<VirtualHost *:80>
ServerName myserver.univ.edu
ServerAlias *.myserver.univ.edu
UseCanonicalName off
DocumentRoot "/var/www/html"
</VirtualHost>
<VirtualHost *:80>
ServerName wormbase.univ.edu
ServerAlias *.wormbase.univ.edu
ProxyPass / http://172.16.236.128/
ProxyPassReverse / http://172.16.236.128/
</VirtualHost>

In the example above, 172.16.236.128 was the IP of the virtual machine. Restart the webserver and the proxying should take effect.

Troubleshooting

Resetting the MAC address

MAC addresses are set automically in virtual machines based on the host OS name and the path to the .vmx file on the host OS. If you move the virtual machine, the MAC address will be incorrect.

When booted into graphical mode:

System Tools > Network

Double click on the network adaptor and select the "Hardware" tab. Click on "Probe", then "OK".