Page:  
May
8

Simple Python Class Library and Command Line Utility for S3 Storage and Retrieval

Hello.  This is based on the Python Boto library.  I created a class with a couple functions for storing and retrieving information (either publicly or privately) on Amazon S3.  I added sufficient error handling to allow this code to be used in a production environment. Improvements and suggestions are always welcome.  It also doubles as a command line utility for simply upload and and download files.  I love that feature about Python.  I wrote this so I could be DRY.  I hope it helps you get your head in the cloud. Enjoy!



#!/usr/bin/env python
#
# simpleS3.py
#
# By: Alan Viars
# Copyright Videntity Systems, Inc. 2009
# All rights Reseved.
# License: New BSD
# Last Updated: May 9, 2009
#
# This was tested using Python 2.5 and Ubuntu Linux, but
# it should run fine w/ other configurations.
# You will need to install boto to get this library running
# and of  course you need an S3 account from Amazon.
# See http://aws.amazon.com
#
# NOTES ON INSTALLING BOTO:
# 1.7.a is latestversion of boto at the time of writing.
# Execute the following from a command line prompt
# > wget http://boto.googlecode.com/files/boto-1.7a.tar.gz
# > tar zxvf boto-1.7a.tar.gz
# > cd boto-1.7a
# Run this as root or w/ admin privileges
# > python setup.py install
# > if on Ubuntu or Debian deravitive, use sudo like so:
# > sudo python setup.py install

"""
simpleS3.py
"""

import sys
from boto.s3.connection import S3Connection
from boto.s3.key import Key
import mimetypes

#Set these to match your Amazon S3 Account
AWS_ACCESS_KEY= '*****PUT_YOUR_KEY_HERE****'
AWS_SECRET_ACCESS_KEY='*****PUT_YOUR_SECRET_KEY_HERE****'

class SimpleS3:
    """
    A very simple class library to simple store
    and  retieve files in Amazon S3
    Works with HTTPS/port 443 only (no HTTP/port 80)
    """

    #Store a file in s3
    def store_in_s3 (bucket, filename,
                     local_filepath,
                     public=False):

            """Store a file in s3"""
            url=""
            try:
                conn= S3Connection(AWS_ACCESS_KEY,
                                   AWS_SECRET_ACCESS_KEY)
                b = conn.create_bucket(bucket)
                k=Key(b)
                k.key=filename

                mime = mimetypes.guess_type(filename)[0]

                if mime==None:
                    print "I couldn't guess MIME because"
                    print "I couldn't detect a file ext."
                    print "Using 'application/octet-stream'"
                    print "as the default MIME instead."
                    mime = "application/octet-stream"

                print "MIME Type = %s" % (mime)
                k.set_metadata("Content-Type", mime)
                k.set_contents_from_filename(local_filepath)
                if public==True:
                    k.set_acl("public-read")
                url = "https://%s.s3.amazonaws.com/%s" % (bucket,
                                                          k.key)
            except:
                print "Error in store_in_s3."
                print sys.exc_info()
                return url
            finally:
                return url

    #Get a file from s3
    def get_from_s3 (bucket, filename, local_filepath ):
            """Get a file from s3"""
            retval = False
            try:
                conn= S3Connection(AWS_ACCESS_KEY,
                                   AWS_SECRET_ACCESS_KEY)

                b = conn.create_bucket(bucket)
                k = Key(b)
                k.key = filename
                k.get_contents_to_filename(local_filepath)
                retval = True
            except:
                print "Error in get_from_s3"
                print sys.exc_info()
                return retval
            finally:
                return retval

    # Our MAIN application which takes 3 command line arguments
    # Take in a mode, bucketname, filename, and public T/F.
    # if mode=PUT, then store the file in S3
    # If mode=GET, then read the file from S3,
    # and write it to local disk

    if __name__ == "__main__":

        try:

            if sys.argv[1]=='PUT':
                mode='PUT'
            elif sys.argv[1]=='GET':
                mode='GET'
            else:
                mode=''   

            if sys.argv[5] in ('t', 'T','True','true','TRUE',):
                public=True
            else:
                public=False

            if mode=='PUT':
                print """Saving %s file to S3 bucket %s.
                Public = %s""" % (sys.argv[3],sys.argv[2],public )

                url = store_in_s3 (sys.argv[2],
                                   sys.argv[3],
                                   sys.argv[4],
                                   public)

                if url != "":
                    print "File saved in S3.  URL is %s" % (url)
                else:
                    print "Error in store_in_s3.  No URL returned."

            elif mode=='GET':
                print "Reading %s from S3 and saving it to to %s" 
                    %(sys.argv[3], sys.argv[4])

                #get the file from S3
                retval = get_from_s3 (sys.argv[2],
                                      sys.argv[3],
                                      sys.argv[4] )

                if retval==True:
                    print "File %s saved locally." % (sys.argv[3])
                else:
                    print "Error getting file from S3 and saving 
                           locally."
            else:
                print "Invalid mode.  Pleas specify PUT or GET."

        except IndexError:
            print "Please supply all parameters in the form:"
            print "simples3.py <GET/PUT> <bucketname> <key> 
            <local_filename> <Public? T/F>"
            print "Example:"
            print "simples3.py PUT mybucket my_key my_local_filename T"
            exit (1)

        except:
            print "Misc. Error. See below."
            print sys.exc_info()
            exit (1)

Download it here : SimpleS3.py

May
1

A Few Handy Resources for Entrepreneurs

Here are several links we discussed at the Canton ETC event. I hope these may be useful to fellow entrepreneurs going through “the struggle”, “the fire”, “the dip”, etc.

Thank you Fulya for organizing.

Apr
15

Removing Asterisk on Ubuntu 6,8

This little recipe works for me if you’re not using any special hardware.

Run these commands as root or use sudo. This will work for Asterisk 1.4 and 1.6.


sudo killall -9 safe_asterisk
sudo killall -9 asterisk
sudo rm -rf /var/log/asterisk
sudo rm -rf /var/lib/asterisk
sudo rm -rf /var/spool/asterisk
sudo rm -rf /usr/lib/asterisk
sudo rm -rf /etc/asterisk
sudo rm /usr/sbin/asterisk
sudo rm /usr/sbin/safe_asterisk

Now you should be clean and ready to reinstall if needed.

Mar
2

Installing MySQL for Python on Mac OS X (MySQLdb)

I was having issues setting up MySQL Python when I was setting up a Django development environment on Mac OS X. This works fine on Ubuntu and Windows Machines, but it appears installing Python’s MySQL (MySQLdb) can be difficult on Mac OS.  See this post http://forums.mysql.com/read.php?50,175059,179979

It tried the following compilation but it ran with some errors:

http://www.keningle.com/?p=11

Anyhow a bunch of people have had problems with this.  Here is what I did to get it to work nice and easy:

Download this file. mysql_python-122-py25-macosx-105-i386.egg Install this egg file to file: to /Library/Python/2.5/site-packages/. So open up a terminal shell and do something like:


sudo cp mysql_python-122-py25-macosx-105-i386.egg /Library/Python/2.5/site-packages/

Now create and edit a text file here /Library/Python/2.5/site-packages/myeggs.pth:


sudo nano /Library/Python/2.5/site-packages/myeggs.pth

Now put the full path to the egg in the file:


/Library/Python/2.5/site-packages/mysql_python-122-py25-macosx-105-i386.egg

Save the file:


ctrl-x
Save changes? Y

That’s it. All should be working. Compiling Python_MySql from scratch should not be necessary. If you are using Django, you can test this with:

python manage.py syncdb

You can also test it just using Python by:

python
>>> import MySQLdb
>>>

If you see no error, then the library is present. If you see “ImportError: No module named MySQLdb”, then something has gone wrong.

Change the above based on your local settings. Happy coding.

-Alan

Feb
26

How to create a MySQL database and users from the command line

Create the database with the mysqladmin tool.
Don’t forget to add the bin folder to your path if you are on Windows or Mac!  Not necessary on Linux.
For example, on my Windows box it is C:Program FilesMySQLMySQL Server 5.1bin

mysqladmin --user=root --pass=some_pass create local_db

(replace “some_pass” with your mysql root password.)

now login to my sql as root..
mysql --user=root --pass=some_pass

now change to the new db…

mysql>use  local_db;
Database changed.
mysql>

create the user(s) and grant rights ….this allows the user monty to access the database locally and remotely.

mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
    ->     IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%'
    ->     IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
Page: