http://swift.openstack.org/development_saio.html

SAIO - Swift All In One

Instructions for setting up a development VM

This documents setting up a virtual machine for doing Swift development. Thevirtual machine will emulate running a four node Swift cluster.

• Get the Ubuntu 10.04 LTS (Lucid Lynx) server image:
  • Ubuntu Server ISO: http://releases.ubuntu.com/lucid/ubuntu-10.04.2-server-amd64.iso (717 MB)
  • Ubuntu Live/Install: http://cdimage.ubuntu.com/releases/lucid/release/ubuntu-10.04.2-dvd-amd64.iso (4.2 GB)
  • Ubuntu Mirrors: https://launchpad.net/ubuntu/+cdmirrors
• Create guest virtual machine from the Ubuntu image.

Additional information about setting up a Swift development snapshot on other distributions isavailable on the wiki at http://wiki.openstack.org/SAIOInstructions.

Installing dependencies and the core code

• As root on guest (you’ll have to log in as you, then sudo su -):
  1. apt-get install python-software-properties
  2. add-apt-repository ppa:swift-core/release
  3. apt-get update
  4. apt-get install curl gcc git-core memcached python-configobjpython-coverage python-dev python-nose python-setuptools python-simplejsonpython-xattr sqlite3 xfsprogs python-webob python-eventletpython-greenlet python-pastedeploy python-netifaces
  5. Install anything else you want, like screen, ssh, vim, etc.
  6. Next, choose either Using a partition for storage or Using a loopback device for storage.

Using a partition for storage

If you are going to use a separate partition for Swift data, be sure to add another device when

creating the VM, and follow these instructions.

1. fdisk /dev/sdb (set up a single partition)

2. mkfs.xfs -i size=1024 /dev/sdb1

3. Edit /etc/fstab and add

   /dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0

4. mkdir /mnt/sdb1

5. mount /mnt/sdb1

6. mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

7. chown : /mnt/sdb1/*

8. mkdir /srv

9. for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

10. mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

11. chown -R : /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/

12. Add to /etc/rc.local (before the exit 0):

    mkdir /var/run/swiftchown : /var/run/swift

13. Next, skip to Setting up rsync.

Using a loopback device for storage

If you want to use a loopback device instead of another partition, follow these instructions.

1. mkdir /srv

2. dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000

   (modify seek to make a larger or smaller partition)

3. mkfs.xfs -i size=1024 /srv/swift-disk

4. Edit /etc/fstab and add

   /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0

5. mkdir /mnt/sdb1

6. mount /mnt/sdb1

7. mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4

8. chown : /mnt/sdb1/*

9. for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

10. mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift

11. chown -R : /etc/swift /srv/[1-4]/ /var/run/swift – Make sure to include the trailing slash after /srv/[1-4]/

12. Add to /etc/rc.local (before the exit 0):

    mkdir /var/run/swiftchown : /var/run/swift

Setting up rsync

1. Create /etc/rsyncd.conf:

   uid = gid = log file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 127.0.0.1[account6012]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/account6012.lock[account6022]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/account6022.lock[account6032]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/account6032.lock[account6042]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/account6042.lock[container6011]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/container6011.lock[container6021]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/container6021.lock[container6031]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/container6031.lock[container6041]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/container6041.lock[object6010]max connections = 25path = /srv/1/node/read only = falselock file = /var/lock/object6010.lock[object6020]max connections = 25path = /srv/2/node/read only = falselock file = /var/lock/object6020.lock[object6030]max connections = 25path = /srv/3/node/read only = falselock file = /var/lock/object6030.lock[object6040]max connections = 25path = /srv/4/node/read only = falselock file = /var/lock/object6040.lock

2. Edit the following line in /etc/default/rsync:

   RSYNC_ENABLE=true

3. service rsync restart

Optional: Setting up rsyslog for individual logging

1. Create /etc/rsyslog.d/10-swift.conf:

   # Uncomment the following to have a log containing all logs together#local1,local2,local3,local4,local5.*   /var/log/swift/all.log# Uncomment the following to have hourly proxy logs for stats processing#$template HourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"#local1.*;local1.!notice ?HourlyProxyLoglocal1.*;local1.!notice /var/log/swift/proxy.loglocal1.notice           /var/log/swift/proxy.errorlocal1.*                ~local2.*;local2.!notice /var/log/swift/storage1.loglocal2.notice           /var/log/swift/storage1.errorlocal2.*                ~local3.*;local3.!notice /var/log/swift/storage2.loglocal3.notice           /var/log/swift/storage2.errorlocal3.*                ~local4.*;local4.!notice /var/log/swift/storage3.loglocal4.notice           /var/log/swift/storage3.errorlocal4.*                ~local5.*;local5.!notice /var/log/swift/storage4.loglocal5.notice           /var/log/swift/storage4.errorlocal5.*                ~

2. Edit /etc/rsyslog.conf and make the following change:

   $PrivDropToGroup adm

3. mkdir -p /var/log/swift/hourly

4. chown -R syslog.adm /var/log/swift

5. service rsyslog restart

Getting the code and setting up test environment

Sample configuration files are provided with all defaults in line-by-line comments.

Do these commands as you on guest.

1. mkdir ~/bin

2. Check out the swift repo with git clone https://github.com/openstack/swift.git

3. Build a development installation of swift, for example:cd ~/swift; sudo python setup.py develop

4. Edit ~/.bashrc and add to the end:

   export SWIFT_TEST_CONFIG_FILE=/etc/swift/func_test.confexport PATH=${PATH}:~/bin

5. . ~/.bashrc

Configuring each node

Sample configuration files are provided with all defaults in line-by-line comments.

1. Create /etc/swift/proxy-server.conf:

   [DEFAULT]bind_port = 8080user = log_facility = LOG_LOCAL1[pipeline:main]pipeline = healthcheck cache tempauth proxy-server[app:proxy-server]use = egg:swift#proxyallow_account_management = trueaccount_autocreate = true[filter:tempauth]use = egg:swift#tempauthuser_admin_admin = admin .admin .reseller_adminuser_test_tester = testing .adminuser_test2_tester2 = testing2 .adminuser_test_tester3 = testing3[filter:healthcheck]use = egg:swift#healthcheck[filter:cache]use = egg:swift#memcache

2. Create /etc/swift/swift.conf:

   [swift-hash]# random unique string that can never change (DO NOT LOSE)swift_hash_path_suffix = changeme

3. Create /etc/swift/account-server/1.conf:

   [DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6012user = log_facility = LOG_LOCAL2[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

4. Create /etc/swift/account-server/2.conf:

   [DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6022user = log_facility = LOG_LOCAL3[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

5. Create /etc/swift/account-server/3.conf:

   [DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6032user = log_facility = LOG_LOCAL4[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

6. Create /etc/swift/account-server/4.conf:

   [DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6042user = log_facility = LOG_LOCAL5[pipeline:main]pipeline = account-server[app:account-server]use = egg:swift#account[account-replicator]vm_test_mode = yes[account-auditor][account-reaper]

7. Create /etc/swift/container-server/1.conf:

   [DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6011user = log_facility = LOG_LOCAL2[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

8. Create /etc/swift/container-server/2.conf:

   [DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6021user = log_facility = LOG_LOCAL3[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

9. Create /etc/swift/container-server/3.conf:

   [DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6031user = log_facility = LOG_LOCAL4[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

10. Create /etc/swift/container-server/4.conf:

    [DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6041user = log_facility = LOG_LOCAL5[pipeline:main]pipeline = container-server[app:container-server]use = egg:swift#container[container-replicator]vm_test_mode = yes[container-updater][container-auditor][container-sync]

11. Create /etc/swift/object-server/1.conf:

    [DEFAULT]devices = /srv/1/nodemount_check = falsebind_port = 6010user = log_facility = LOG_LOCAL2[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

12. Create /etc/swift/object-server/2.conf:

    [DEFAULT]devices = /srv/2/nodemount_check = falsebind_port = 6020user = log_facility = LOG_LOCAL3[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

13. Create /etc/swift/object-server/3.conf:

    [DEFAULT]devices = /srv/3/nodemount_check = falsebind_port = 6030user = log_facility = LOG_LOCAL4[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

14. Create /etc/swift/object-server/4.conf:

    [DEFAULT]devices = /srv/4/nodemount_check = falsebind_port = 6040user = log_facility = LOG_LOCAL5[pipeline:main]pipeline = object-server[app:object-server]use = egg:swift#object[object-replicator]vm_test_mode = yes[object-updater][object-auditor]

Setting up scripts for running Swift

1. Create ~/bin/resetswift.

   If you are using a loopback device substitute /dev/sdb1 with /srv/swift-disk.

   If you did not set up rsyslog for individual logging, remove the find /var/log/swift... line:

   #!/bin/bashswift-init all stopfind /var/log/swift -type f -exec rm -f {} \;sudo umount /mnt/sdb1sudo mkfs.xfs -f -i size=1024 /dev/sdb1sudo mount /mnt/sdb1sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4sudo chown : /mnt/sdb1/*mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslogsudo service rsyslog restartsudo service memcached restart

2. Create ~/bin/remakerings:

   #!/bin/bashcd /etc/swiftrm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gzswift-ring-builder object.builder create 18 3 1swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1swift-ring-builder object.builder rebalanceswift-ring-builder container.builder create 18 3 1swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1swift-ring-builder container.builder rebalanceswift-ring-builder account.builder create 18 3 1swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1swift-ring-builder account.builder rebalance

3. Create ~/bin/startmain:

   #!/bin/bashswift-init main start

4. Create ~/bin/startrest:

   #!/bin/bashswift-init rest start

5. chmod +x ~/bin/*

6. remakerings

7. cd ~/swift; ./.unittests

8. startmain (The Unable to increase file descriptor limit.  Running as non-root? warnings are expected and ok.)

9. Get an X-Storage-Url and X-Auth-Token: curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0

10. Check that you can GET account: curl -v -H 'X-Auth-Token: '

11. Check that swift works: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat

12. cp ~/swift/test/functional/sample.conf /etc/swift/func_test.conf

13. cd ~/swift; ./.functests (Note: functional tests will first deleteeverything in the configured accounts.)

14. cd ~/swift; ./.probetests (Note: probe tests will reset yourenvironment as they call resetswift for each test.)

If you plan to work on documentation (and who doesn’t?!):

On Ubuntu:

1. sudo apt-get install python-sphinx installs Sphinx.
2. python setup.py build_sphinx builds the documentation.

On MacOS:

1. sudo easy_install -U sphinx installs Sphinx.
2. python setup.py build_sphinx builds the documentation.

Debugging Issues

If all doesn’t go as planned, and tests fail, or you can’t auth, or something doesn’t work, here are some good starting places to look for issues:

1. Everything is logged in /var/log/syslog, so that is a good first place tolook for errors (most likely python tracebacks).
2. Make sure all of the server processes are running. For the basefunctionality, the Proxy, Account, Container, and Object serversshould be running
3. If one of the servers are not running, and no errors are logged to syslog,it may be useful to try to start the server manually, for example:swift-object-server /etc/swift/object-server/1.conf will start theobject server. If there are problems not showing up in syslog,then you will likely see the traceback on startup.