Installation of this client is very similar to the existing steps to install OpenStack clients, but I will put my methods of doing it here for quick reference.

Quick Install Steps (the TL;DR)

This will create a new directory, install the client and extensions, as well as give you a little file to start working on for the profile.

If you want to be safe use this to install into a virtualenv:

[code lang=text]
mkdir neutronclient && cd neutronclient
virtualenv –prompt='(neutronclient)' .venv
source .venv/bin/activate
pip install rackspace-neutronclient
wget http://tinyurl.com/neutron-profile
[/code]

If you don’t want to be safe and fill your system with random things:

[code lang=text]
sudo pip install rackspace-neutronclient
wget http://tinyurl.com/neutron-profile
[/code]

Make edits to the neutron-profile file you just downloaded and you should be set to go.

Installing the Client with Extensions

Assuming you’re in a virtualenv, which I highly recommend, you can just run these commands without using sudo:

[code lang=text]
pip install rackspace-neutronclient
[/code]

Using the client

Because we didn’t rename the binary installed by the package the client will still be named neutron. This may cause some issues (see Troubleshooting).

For these examples os-auth-url needs to be given.

Example: Creating a network

[code lang=text]
neutron –os-auth-url https://identity.api.rackspacecloud.com/v2.0 –os-auth-strategy rackspace –os-username USERNAME –os-password API_KEY net-create test_net_name
[/code]

Example: Listing Networks

[code lang=text]
neutron –os-auth-url https://identity.api.rackspacecloud.com/v2.0 –os-auth-strategy rackspace –os-username USERNAME –os-password API_KEY net-list
[/code]

Using the client with Environment Args

If you don’t want to pass the arguments to the client as above you may also put all that information into your environment. This is very similar to the way it works with the nova client.

Create a new file or put the following into your ~/.bash_profile:

[code lang=text]
export OS_AUTH_URL=https://identity.api.rackspacecloud.com/v2.0/
export OS_AUTH_STRATEGY=rackspace
export OS_REGION_NAME=DFW
export OS_USERNAME=<username>
export OS_TENANT_NAME=<tenant_id>
export OS_PROJECT_ID=<tenant_id>
export OS_PASSWORD=<api_key>
export OS_NO_CACHE=1
[/code]

GOTCHA: OS_AUTH_STRATEGY entry is different than OS_AUTH_SYSTEM.

Then source the file (or ~/.bash_profile). You should now be able to do the above commands like so:

Example: Creating a network

[code lang=text]
neutron net-create test_net_name
[/code]

Example: Listing Networks

[code lang=text]
neutron net-list
[/code]

Using the Client with supernova

The instructions on how to do this were taken from here. If you’ve never used supernova before:

  1. You really should
  2. You should read how to use it before continuing

So you’re a supernova expert now? Ok. You should have a .supernova file. Insert into that file a new section like so (note: this is the most of the above script without the exports):

[code lang=text]
[neutron-dfw]
OS_EXECUTABLE=neutron
OS_AUTH_URL=https://identity.api.rackspacecloud.com/v2.0/
OS_AUTH_STRATEGY=rackspace
OS_REGION_NAME=DFW
OS_USERNAME=<username>
OS_TENANT_NAME=<tenant_id>
OS_PROJECT_ID=<tenant_id>
OS_PASSWORD=<api_key>
OS_NO_CACHE=1
[/code]

The relevant part is the OS_EXECUTABLE entry.

GOTCHA: OS_AUTH_STRATEGY entry is different than OS_AUTH_SYSTEM.

You should now be able to use supernova.

Example: Creating a network

[code lang=text]
supernova neutron-dfw net-create test_net_name
[/code]

Example: Listing Networks

[code lang=text]
supernova neutron-dfw net-list
[/code]

Troubleshooting

The client is upgraded often, so if you’re running into any troubles the first thing you want to do is (prefix sudo if necessary):

[code lang=text]
pip install –upgrade rackspace-neutronclient
[/code]

Otherwise, a vast majority of the problems encountered are due to having python-neutronclient (upstream client) and rackspace-python-neutronclient (forked client) installed at the same time.

If you are having any problems first try the following and reinstall using the above instructions:

[code lang=text]
pip uninstall -y python-neutronclient
pip uninstall -y rackspace-neutronclient
pip uninstall -y rackspace-auth-neutronclientext
[/code]

When both are installed, what is happening?

Because the forked client uses the exact code as the upstream code they are placed into the site-packages (libs) in the same place. The one installed last will be what is used. Even more troublesome is that if you uninstall one of the clients you are removing code that both depend on. Always uninstall both if having problems.

Our neutron plugin quark accepts many drivers and will use however many that are configured at the same time. Quark currently has no default working configuration in which it can be tested.

This setup will create a neutron server that really doesn’t do anything, but it will not require any other services to run! It also assumes that you have a mysql server configured. If you don’t want to use mysql you can change the sqlalchemy connection string in the neutron.conf provided at the end of the post.

Installing the Packages

First thing we need to do is download the code somewhere and install it. The things we really need to install are:

Create a directory somewhere. This will be where we install all the packages. We will be using virtualenv because we’re good python developers, aren’t we?

[code lang=text]
cd my_directory_of_choice
virtualenv –prompt='(quark)' .venv
source .venv/bin/activate
[/code]

Your shell prompt should, at least if you’re using bash, have (quark) in front now.

The rest of this is pretty easy. Since we’re looking to develop quark/neutron we will need source and not packages.

[code lang=text]
git clone git@github.com:openstack/neutron && cd neutron && python setup.py develop
git clone git@github.com:rackerlabs/quark && cd quark && python setup.py develop
[/code]

If you run pip freeze you should see some references to neutron and quark in there. There will most likely be a bunch of other packages installed that are dependencies for neutron or quark.

Configuration Details

Make a configuration directory. I call mine etc for giggles. See the full file at the end of this to copy-pasta.

My default testing configuration uses our unmanaged driver. This driver is essentially a noop driver and doesn’t require any extra software to be installed to use quark. Caveat: because it is noop you will have no connectivity if placed into an environment.

Put the following configuration into your neutron.conf (I usually put it at the bottom so the config grouping doesn’t get confused).

[code lang=text]
[QUARK]
net_driver=quark.drivers.unmanaged.UnmanagedDriver
default_network_type=UNMANAGED

[/code]

  • set the core_plugin value in the neutron.conf to quark.plugin.Plugin.
  • setting the auth_strategy to noauth is very helpful especially if you are using wafflehaus.try_context to fake out DB connections

Creating the Database

You’ll need to setup a database (probably mysql). Then modify your neutron.conf with the appropriate connection string (see this). To create the initial quark database you need to run:

[code lang=text]
quark-db-manage –config-file <path-to>neutron.conf upgrade head
[/code]

Creating the Default Network Artifacts

This resty call will create the mac address ranges for this simple setup:

[code lang=text]
POST /v2.0/mac_address_ranges.json -d '{"mac_address_range": {"cidr" : "AA:BB:CC", "tenant_id": "provider"}}'
[/code]

And thisresty call will create the public network:

[code lang=text]
POST /v2.0/networks -d '{"network": {"id": "00000000-0000-0000-0000-000000000000", "tenant_id": "provider", "name": "public"}}'
[/code]

Finally you will need to make this resty call to make a subnet.

[code lang=text]
POST /v2.0/subnets -d '{"subnet": {"network_id": "00000000-0000-0000-0000-000000000000", "segment_id": "blah", "cidr": "10.0.0.0/16", "tenant_id": "derp", "ip_version": "4"}}'
[/code]

Isolated (tenant owned) networks are currently not supported in this configuration. A more operative driver is necessary. There is an NVP/NSX driver included with quark but it requires standing up a NVP/NSX Controller. Doing such a thing is well beyond the scope of this post.

Running Neutron

Then to run the neutron server locally you run the command (preferably in a virtualenv):

[code lang=text]
neutron-server –config-dir /home/jhammond/quark_server/etc
[/code]

Usage and Initial Smoke Test

Some useful tidbits of information about how we’ve changed how things work in quark:

  • to support cells we had to add segment_id to subnets on provider networks
  • ports created on provider networks require a segment_id as well
  • during testing this value can be anything as long as they match

This script uses resty because I dislike curl. Note the segment id in the POST subnets and POST ports.

[code lang=text]
#!/bin/bash
endpoint="http://localhost:9696/v2.0"
tenant='random'
net_name='test_network'
source ~/bin/resty/resty -W $endpoint -H "Content-type: application/json"
net_id='00000000-0000-0000-0000-000000000000'

subnet=`POST /subnets -d '{"subnet": {"network_id": "'$net_id'", "tenant_id": "'$tenant'", "cidr": "192.168.7.1/24", "ip_version": 4, "segment_id":
"test"}}'`

subnet_id=`echo $subnet | python -c 'import sys, json; print json.load(sys.stdin)["subnet"]["id"]'`
echo "Made subnet $subnet_id"

port=`POST /ports -d '{"port": {"tenant_id": "'$tenant'", "name": "weeble", "network_id": "'$net_id'", "segment_id": "test", "fixed_ips": [{"ip_address":
"192.168.7.50", "subnet_id": "'$subnet_id'"}]}}'`

port_id=`echo $port | python -c 'import sys, json; print json.load(sys.stdin)["port"]["id"]'`
echo "Made port $port_id"

# CLEAN-UP
DELETE /ports/$port_id
DELETE /subnets/$subnet_id
GET /networks
[/code]

My etc Contents

[code lang=text]
api-paste.ini
dhcp_agent.ini
fwaas_driver.ini
init.d/
l3_agent.ini
lbaas_agent.ini
metadata_agent.ini
metering_agent.ini
neutron/
neutron.conf
policy.json
rootwrap.conf
services.conf
vpn_agent.ini
[/code]

Full neutron.conf

Fill out the database connection info. If you remove the database info completely sqlalchemy will attempt to connect to a mysql server with the following connection string:

mysql://root:password@127.0.0.1/neutron?charset=utf8

That may work for you, but you should probably change it.

[code lang=text]
[DEFAULT]
lock_path = $state_path/lock
core_plugin = quark.plugin.Plugin
auth_strategy = noauth
fake_rabbit = true
rpc_backend = fake

[DATABASE]
connection = mysql://db_user:db_password@127.0.0.1/neutron?charset=utf8
reconnect_interval = 5
auto_create_schema = False
min_pool_size = 100
max_pool_size = 500
max_overflow = 500

[QUARK]
net_driver=quark.drivers.unmanaged.UnmanagedDriver
default_net_strategy='{"00000000-0000-0000-0000-000000000000": {"bridge": "publicnet"}, "11111111-1111-1111-1111-111111111111": {"bridge": "servicenet"}}'
default_network_type=UNMANAGED
# IPAM recycling interval, in seconds
ipam_reuse_after= 300
ipam_retry=1
show_subnet_ip_policy_id = False
show_allocation_pools = True
pessimistic_connection_pooling = True
environment_capabilities = ""
[/code]

Troubleshooting

TypeError: %d format: a number is required, not NoneType

This is caused by oslo_messaging and rpc functions I do not use. Ensure you have the config variable rpc_backend = fake. This will disable that feature.

Running the Quark Unit Tests

We use tox to run the unit tests. A gotcha with quark is that it has assumed configurations for local mysql DB testing (tox -e mysql). If your local DB doesn’t match these assumptions they will fail.

The following will allow those to work:

[code lang=text]
QUARK_MYSQL_TESTS_URL="mysql://user:password@localhost/quark_functional_tests" tox -e mysql
[/code]

This will produce a lot of sqlalchemy output. I use this little snippet to keep the output to something sane:

QUARK_MYSQL_TESTS_URL=&quot;mysql://user:password@localhost/quark_functional_tests&quot; tox -e mysql 2&gt;&amp;1 &gt;/dev/null | grep -B 25 'end captured stdout'Getting OpenStack neutron+quark Working Locally

For many reasons we’ve found it necessary to upload a fork of python-neutronclient. I do this so rarely that I always forget how to do it and this post is here to remind me. Hopefully other people can get some help from this.

Changing up the setup.cfg

Because this is a fork we need to change a couple things (otherwise it won’t be a fork anymore). The biggest things to change are the name, author, author-email, and home-page.

[code lang=text]
name = rackspace-python-neutronclient
author = Rackspace
author-email = neutron-requests@lists.rackspace.com
home-page = http://www.rackspace.com
[/code]

Commit these changes and push to whatever branch you need to. I push here.

Preventing the PBR “git installed?” Problem

This is caused by version.py looking for the wrong distribution name. Simply modify python-neutronclient to rackspace-python-neutronclient:

[code lang=text]
target = 'rackspace-python-neutronclient'
__version__ = pbr.version.VersionInfo(target).version_string()
[/code]

Setting the Version with Tags (no longer necessary)

This used to be the only known way to get sdist to populate the version in PKG-INFO. It is now recommended to set the version with an environment variable passed during the upload step. This is here just in case that trick stops working.

When you attempt to upload to pypi it fails with random reasons. It appears that some part of the setup process requires for there to be a tag on the commit you’re currently on.

Create a tag with the version using a X.Y.Z format:

[code lang=text]
git tag -a 2.4.1 -m "Rackspace OpenStack Network Client"
[/code]

Now upload to pypitest then pypi

If this part is confusing check out this guide on pypiy before continuing. I assume by this point you have pypitest and pypi setup as target repositories.

If your tag is setup correctly all you should need to do is run the following:

[code lang=text]
PBR_VERSION=x.y.z python setup.py sdist upload -r pypitest
[/code]

You should be able to goto testpypi and find your package at this point. If the version looks like what you want then you can continue to upload to the real pypi. At one point I had weird additional things appended to my version. The solution to this was to delete the tag, git tag -d, recreate it as above, then upload it again to pypitest.

To upload to pypi run:

[code lang=text]
PBR_VERSION=x.y.z python setup.py sdist upload -r pypi
[/code]

That should be it!

Versioning Silliness

Because of the way that pypi works, the version that you choose during the upload step must be different than any version you have ever loaded. There is no known way to get beyond this only-append feature of pypi.

If anyone knows how to get beyond this please leave a comment. It is really annoying.

Backstory

I really wanted to have the filename of the file I was editing to be a different color than the path of the file. Using statusline without any tricks would only provide just the filename or the full path with filename.

Solution

bufname and fnamemodify to the rescue!

Using bufname with '%' as an argument: bufname('%') will return the filename of file currently in the buffer.

Using fnamemodify with the above and the ':p:h' as escape codes will give you the entire path of the passed file and then cut off the filename.

All together it looks like: fnamemodify(bufname('%'), ':p:h')

My full statusline (with helpful comments)

[code lang=text]
" put paste info
set statusline=%#error#%{SL('HasPaste')}%*
" set color to 'search'
set statusline+=%#todo#
" if a help file or not
set statusline+=%h
" if modified or not
set statusline+=%m
" set color to 'error'
set statusline+=%#error#
" read-only or not
set statusline+=%r
" reset color
set statusline+=%*
" FILETYPE (COMMENTED OUT FOR NOW)
"set statusline+=%y\
" column number
set statusline+=\ %c,
" line/total lines
set statusline+=%l/%L
" percent of file
set statusline+=\ %P
" Move to the right align
set statusline+=%=
" put up to 50 characters of filename and path
set statusline+=\ %30.30{fnamemodify(bufname('%'),':p:h')}/
set statusline+=%#statusbold#
set statusline+=%t
" space at end to make it easier to read
set statusline+=\
[/code]

Some other tips:

To add a space you need to use: \_ where _ is actually a space.
You can define your own highlight names (seen above with #statusbold#) with:

[code lang=text]
highlight statusbold term=bold,reverse ctermfg=5 ctermbg=118 gui=bold,reverse
[/code]

We’ll be using a Ubuntu 14.10 image. The size/flavor you choose depends on the amount of players that you want to support.

Synopsis

  • Make instance
  • Install stuff and run Minecraft in tmux

Process of Making a Rackspace Instance

[code lang=text]
<wip>
[/code]

Installing the Base Packages

[code lang=text]
# update all of our package lists
sudo apt-get update

# install utilities and java
# if openjdk-7-jre is not available use openjdk-6-jre
sudo apt-get -y install unzip python-software-properties \
openjdk-7-jre tmux

# create a user to run the server as and become them
sudo adduser minecraft
sudo su minecraft
cd ~

# get the config files for the server to work
cd /usr/local
sudo wget http://mcmyadmin.com/Downloads/etc.zip
sudo unzip etc.zip

# download mcmyadmin and unzip
wget http://mcmyadmin.com/Downloads/MCMA2_glibc25.zip
unzip MCMA2_glibc25.zip && rm MCMA2_glibc25.zip

# create our working directory
mkdir mcmyadmin
mv MCMA2_Linux_x86_64 mcmyadmin

# run the script with the admin password
# this will put you into blue(maybe) screen saying to
# complete setup with the web interface
./MCMA2_Linux_x86_64 -setpass A_Password_For_You

# You should now be able to log on to the web interface
# which is at http://YOUR_IP:8080
# I recommend setting your License if you have a McMyAdmin key
# You can find the License portion in About > Updates

# When done stop McMyAdmin by typing in the blue interface
/quit

# Then start it again and go to website to accept EULA
./MCMA2_Linux_x86_64

# Your new Minecraft server and world will be up now
[/code]

If you want to import a world

[code lang=text]
# Stop the server from interface using /quit
# These instructions are from some other linux server to this one
# (this might take awhile, my world was 750 MB)
scp world_backup.tar minecraft@YOUR_IP:~

# remove the existing world
rm mcmyadmin/Minecraft/world -rf

# untar and mv the world folder into the Minecraft folder
tar -xvf world_backup.tar
mv world mcmyadmin/Minecraft/

[/code]

Running the Server in Tmux

[code lang=text]
# As the minecraft user:
tmux
cd ~/mcmyadmin
./MCMA2_Linux_x86_64
[/code]

This is apparently a pretty common thing to do but I was unaware. It’s pretty simple once you know about it and I’m just blogging about it to remember:

[code lang=”text”]
# SSH into the machine with the -A argument:

$ ssh -A user@host -p 314

# Once on the machine if you need to use sudo
# use the -E argument to save the environment:

$ sudo -E su
[/code]

Some concerns are that a clever person on the machine can use your credentials to do nefarious things.

This is useful for allowing you to use your ssh-keys (such as github, or SSHing into other machines using a key) from your local machine on remote machines.

You will need to ensure that your ssh config ~/.ssh/config contains (I put mine at the top) ForwardAgent yes for this to work.

Cheers

(Update) The Real Problem and Solution

While creating the fork of the python-neutronclient we ran into this problem again. We found that it was simply that the version.py that was being used was looking for the wrong client. It was a problem with the distribution and not the installation.

I made a post about it.

If you run into this problem this may not be a solution anymore!

Old Solution

I was having a problem with an application I just uploaded to pypi. After I pip installed it to my system I got this error when I ran it:

[code lang=”text”]
$ neutron –version
Traceback (most recent call last):
File &quot;/usr/local/bin/neutron&quot;, line 6, in &lt;module&gt;
from neutronclient.shell import main
File &quot;/usr/local/lib/python2.7/dist-packages/neutronclient/shell.py&quot;, line 73, in &lt;module&gt;
from neutronclient.version import __version__
File &quot;/usr/local/lib/python2.7/dist-packages/neutronclient/version.py&quot;, line 22, in &lt;module&gt;
__version__ = pbr.version.VersionInfo('python-neutronclient').version_string()
File &quot;/usr/local/lib/python2.7/dist-packages/pbr/version.py&quot;, line 78, in version_string
for part in self.release_string().split('.'):
File &quot;/usr/local/lib/python2.7/dist-packages/pbr/version.py&quot;, line 70, in release_string
self.release = self._get_version_from_pkg_resources()
File &quot;/usr/local/lib/python2.7/dist-packages/pbr/version.py&quot;, line 62, in _get_version_from_pkg_resources
return packaging.get_version(self.package)
File &quot;/usr/local/lib/python2.7/dist-packages/pbr/packaging.py&quot;, line 870, in get_version
raise Exception(&quot;Versioning for this project requires either an sdist&quot;
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?
[/code]

It was a weird error and I quickly learned that it was because pbr expects the git tag to have a v in it. Such as v2.3.7. I provided 2.3.7 and that is what caused the bug.

I simply made a new tag (git tag -a v.2.3.7 -m "yey version") and uploaded to pypi (python setup.py sdist upload -r pypi).
Problem with pbr and “sdist tarball”

You need some machine (vm or instance) that has at least 4GB of RAM. 8GB is much better. I will assume you just booted a machine and have root, and nothing else.

Installing Devstack

  • I prefer to make an account (my own) and run devstack as that account (some say to make a stack account, it’s the same thing)
  • adduser herpderp &amp;&amp; adduser herpderp sudo &amp;&amp; apt-get update &amp;&amp; apt-get install git &amp;&amp; exit
  • Log back into the machine as your user; herpderp in this case
  • git clone https://github.com/openstack-dev/devstack.git
  • cd devstack
  • create a new file called localrc with the contents:

[code lang=text]
# Originally from http://www.sebastien-han.fr/blog/2013/08/08/devstack-in-1-minute/
# Misc
DATABASE_PASSWORD=password
ADMIN_PASSWORD=password
SERVICE_PASSWORD=password
SERVICE_TOKEN=password
RABBIT_PASSWORD=password

# Enable Logging

LOGFILE=/opt/stack/logs/stack.sh.log
VERBOSE=True
LOG_COLOR=True
SCREEN_LOGDIR=/opt/stack/logs

# Pre-requisite
ENABLED_SERVICES=rabbit,mysql,key

# Horizon (always use the trunk)
ENABLED_SERVICES+=,horizon
HORIZON_REPO=https://github.com/openstack/horizon
HORIZON_BRANCH=master

# Nova
ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch
IMAGE_URLS+=",https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img"

# Glance
ENABLED_SERVICES+=,g-api,g-reg

# Neutron
ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron,q-lbaas

# Cinder
ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch

# Tempest
ENABLED_SERVICES+=,tempest
[/code]

  • ./stack.sh

Using Devstack

  • Devstack runs inside of a screen session which you access using:
    screen -r
  • Basic screen usage is to hit the escape sequence then some key; the default escape sequence is ctrl+a:
    Example: ctrl+a let go of keys then
  • To move between the many windows in the devstack screen use:
    ctrl+a then " (that’s double quote, or shift+’)

Tips

  • devstack provides unstack.sh and rejoin_stack.sh (or restack.sh); I personally don’t have a lot of luck with these and just recreate the whole machine