More popular than stellar consumer credit you cash advance cash advance your car get your mortgage.


Archive for the ‘coding’ Category

Photos on S3

Friday, October 13th, 2006

You can now see the large version of images without waiting for my slow, slow cable modem async upload. In fact, all the image loads should be blazing fast as soon as I get them all uploaded. At the rate it has been going, it will take 3 days to upload everything to S3. My script is just about though my china pictures now.

And it is so easy to use. I work at Amazon and have had pretty much constant exposure to S3, but it still surprised me how easy it was to use. I used rSh3ell to get started. I just created a bucket named ‘carrino_albums’, then started uploading things to that bucket with acl ‘public-read’.

Here is the script I used to upload my files. Its pretty hacky. I just reads from a file called input and writes the file with that key to S3 as public-read. Some exception handling is thrown in there. I intend on creating a much better script along the lines of s3sync, which behaves like rsync. S3 makes this easy, becasue it automatically stores the upload date and the md5sum in the metadata, and you can retreive that with a HEAD HTTP call. You can also do a list call on the bucket, which will get you the metadata of all the keys in that bucket.

If I upload a file named china_day2/dsc_2572.sized.jpg to bucket carrino_albums, you can find it here. It’s that easy!

#!/usr/bin/env ruby
require 'S3'

AWS_ACCESS_KEY_ID = 'public key'
AWS_SECRET_ACCESS_KEY = 'private key'
USE_SSL = false
@conn = S3::AWSAuthConnection.new(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, USE_SSL)
@bucket = 'carrino_albums'

def write_file(file_name)
    begin
        bytes = nil
        key = file_name  # NOTE: this is specific to our situation
        File.open(file_name, "rb") {|f| bytes = f.read }
        headers = {
              'x-amz-acl' => "public-read",
              'Content-Length' =>  FileTest.size(file_name).to_s
        }
        response = @conn.put(@bucket, key, bytes, headers).http_response.message
        if response == 'OK'
            puts "#{file_name} SUCCESSFULLY uploaded"
        else
            puts "#{file_name} FAILED to be uploaded"
        end
    rescue Interrupt => e
        puts "Caught Interrupt, bailing out"
        exit 1
    rescue Exception => e
        puts "#{file_name} FAILED to be uploaded and caused an exception. #{e.class.name}: #{e.to_s}"
    end
end


STDOUT.sync = true
File.readlines("input").each { |line|
    line.chomp!
    write_file(line)
}

Powerset in Ruby

Friday, August 11th, 2006

Now for a real post in a very cool language that is on the rise. Currently my favorite language to write, Ruby.

Simple PowerSet implementation, pretty straight-forward.

class Array
  def powerSet!
    return [[]] if empty?()
    f = shift()
    rec = powerSet!
    return rec + rec.collect {|i| [f] + i }
  end

  def powerSet
    return clone().powerSet!
  end
end

Note, I added this directly to the Array class. In ruby, classes are left open, so you can add to them later. It is a very dynamic language, and there are hooks for just about everything, so you could have some behavior run when a method is added to your class. Now that Array has a new method, you can just call

[1,2,3].powerSet

Now let’s write it using inject, because inject is awesome. (inject is known as reduce in python, and they are removing it from the next version. It’s a shame they are removing it, but it makes sense because python doesn’t sweet sweet blocks like ruby.)

array = [1,2,3]
array.inject([[]]) { |acc, you|
  ret = []
  acc.each { |i|
    ret << i
    ret << i + [you]
 }
  ret
}

The best part about inject is that once you get your head around it, it is pretty clear and also that this expression can be smashed to 50 chars if that is what gets you going.

[1,2,3].inject([[]]){|c,y|r=[];c.each{|i|r<<i;r<<i+[y]};r}