Put Stuff in Your Windows Azure Junk Trunk – Repository Base

Alright, so the title is rather stupid, but hey, it’s fun!  :)

This project I setup to provide some basic functionality with Windows Azure Storage.  I wanted to use each of the three mediums;  Table, Blob, and Queue, and this example will cover each of these things.  The application will upload and store images, provide a listing, some worker processing, and deletion of the images & associated metadata.  This entry is part 1 of this series, with the following schedule for subsequent entries:

Title aside, schedule laid out, description of the project completed, I’ll dive right in!

Putting Stuff in Your Junk Trunk

Create a new Windows Azure Project called PutJunkInIt.  (Click any screenshot for the full size, and also note some of the text may be off – I had to recreate a number of these images)

Windows Azure PutJunkInIt

Windows Azure PutJunkInIt

Next select the ASP.NET MVC 2 Web Application and also a Worker Role and name the projects JunkTrunk and JunkTrunk.WorkerRole.

Choosing Windows Azure Projects

Choosing Windows Azure Projects

In the next dialog choose to create the unit test project and click OK.

Create Unit Test Project

Create Unit Test Project

After the project is created the following projects are setup within the PutJunkInIt Solution.  There should be a JunkTrunk, JunkTrunk.Worker, JunkTrunk Windows Azure Deployment Project, and a JunkTrunk.Tests Project.

Solution Explorer

Solution Explorer

Next add a Windows Class Library Project and title it JunkTrunk.Storage.

Windows Class Library

Windows Class Library

Add a reference to the Microsoft.WindowsAzure.ServiceRuntime and Microsoft.WindowsAzure.StorageClient assemblies to the JunkTrunk.Storage Project.  Rename the Class1.cs file and class to JunkTrunkBase.  Now open up the Class1.cs file in the JunkTrunk.Storage Project.  First add the following fields and constructor to the class.

public const string QueueName = "metadataqueue";
public const string BlobContainerName = "photos";
public const string TableName = "MetaData";
static JunkTrunkBase()
{
    CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
    {
        configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
        RoleEnvironment.Changed
            += (sender, arg) =>
                    {
                        if (!arg.Changes.OfType()
                                .Any(change => (change.ConfigurationSettingName == configName)))
                            return;
                        if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
                        {
                            RoleEnvironment.RequestRecycle();
                        }
                    };
    });
}

After that add the following blob container and reference methods.

protected static CloudBlobContainer Blob
{
    get { return BlobClient.GetContainerReference(BlobContainerName); }
}
private static CloudBlobClient BlobClient
{
    get
    {
        return Account.CreateCloudBlobClient();
    }
}

Now add code for the table & queue client and reference methods.

protected static CloudQueue Queue
{
    get { return QueueClient.GetQueueReference(QueueName); }
}
private static CloudQueueClient QueueClient
{
    get { return Account.CreateCloudQueueClient(); }
}
protected static CloudTableClient Table
{
    get { return Account.CreateCloudTableClient(); }
}
protected static CloudStorageAccount Account
{
    get
    {
        return
            CloudStorageAccount
            .FromConfigurationSetting("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
    }
}

This class now provides the basic underpinnings needed to retrieve the appropriate information from the configuration.  This base class can then provide that connection information to connect to the table, queue, or blob mediums.

Next step is to create some initialization code to get the containers created if they don’t exist in Windows Azure.  Add a new class file to the PutJunkInIt Project.

JunkTrunkSetup

JunkTrunkSetup

public class JunkTrunkSetup : JunkTrunkBase
{
    public static void CreateContainersQueuesTables()
    {
        Blob.CreateIfNotExist();
        Queue.CreateIfNotExist();
        Table.CreateTableIfNotExist(TableName);
    }
}

Next add the System.Data.Services.Client Assembly to the project.  After adding the assembly add two new classes and name them BlobMeta.cs and Table.cs. Add the following code to the Table.cs Class.

public class Table
{
    public static string PartitionKey;
}

Next add another class file and name it BlobMetaContext.cs and add the following code.

public class BlobMetaContext : TableServiceContext
{
    public BlobMetaContext(string baseAddress, StorageCredentials credentials)
        : base(baseAddress, credentials)
    {
        IgnoreResourceNotFoundException = true;
    }
    public IQueryable Data
    {
        get { return CreateQuery(RepositoryBase.TableName); }
    }
    public void Add(BlobMeta data)
    {
        data.RowKey = data.RowKey.Replace("/", "_");
        BlobMeta original = (from e in Data
                                where e.RowKey == data.RowKey
                                    && e.PartitionKey == Table.PartitionKey
                                select e).FirstOrDefault();
        if (original != null)
        {
            Update(original, data);
        }
        else
        {
            AddObject(RepositoryBase.TableName, data);
        }
        SaveChanges();
    }
    public void Update(BlobMeta original, BlobMeta data)
    {
        original.Date = data.Date;
        original.ResourceUri = data.ResourceUri;
        UpdateObject(original);
        SaveChanges();
    }
}

Now add the following code to the BlobMeta Class.

public class BlobMeta : TableServiceEntity
{
    public BlobMeta()
    {
        PartitionKey = Table.PartitionKey;
    }
    public DateTime Date { get; set; }
    public string ResourceUri { get; set; }
}

At this point, everything should build. Give it a go to be sure nothing got keyed in wrong (or copied in wrong). Once assured the build is still solid, add the Blob.cs Class to the project.

public class Blob : JunkTrunkBase
{
    public static string PutBlob(Stream stream, string fileName)
    {
        var blobRef = Blob.GetBlobReference(fileName);
        blobRef.UploadFromStream(stream);
        return blobRef.Uri.ToString();
    }
    public static Stream GetBlob(string blobAddress)
    {
        var stream = new MemoryStream();
        Blob.GetBlobReference(blobAddress)
            .DownloadToStream(stream);
        return stream;
    }
    public static Dictionary<string, string> GetBlobList()
    {
        var blobs = Blob.ListBlobs();
        var blobDictionary =
            blobs.ToDictionary(
                listBlobItem => listBlobItem.Uri.ToString(),
                listBlobItem => listBlobItem.Uri.ToString());
        return blobDictionary;
    }
    public static void DeleteBlob(string blobAddress)
    {
        Blob.GetBlobReference(blobAddress).DeleteIfExists();
    }
}

After that finalize the Table Class with the following changes and additions.

public class Table : RepositoryBase
{
    public const string PartitionKey = "BlobMeta";
    public static void Add(BlobMeta data)
    {
        Context.Add(data);
    }
    public static BlobMeta GetMetaData(Guid key)
    {
        return (from e in Context.Data
                where e.RowKey == key.ToString() &&
                e.PartitionKey == PartitionKey
                select e).SingleOrDefault();
    }
    public static void DeleteMetaDataAndBlob(Guid key)
    {
        var ctxt = new BlobMetaContext(
            Account.TableEndpoint.AbsoluteUri,
            Account.Credentials);
        var entity = (from e in ctxt.Data
                        where e.RowKey == key.ToString() &&
                        e.PartitionKey == PartitionKey
                        select e).SingleOrDefault();
        ctxt.DeleteObject(entity);
        Repository.Blob.DeleteBlob(entity.ResourceUri);
        ctxt.SaveChanges();
    }
    public static List<BlobMeta> GetAll()
    {
        return (from e in Context.Data
                select e).ToList();
    }
    public static BlobMetaContext Context
    {
        get
        {
            return new BlobMetaContext(
                Account.TableEndpoint.AbsoluteUri,
                Account.Credentials);
        }
    }
}

The final file to add is the Queue.cs Class File. Add that and then add the following code to the class.

public class Queue : JunkTrunkBase
{
    public static void Add(CloudQueueMessage msg)
    {
        Queue.AddMessage(msg);
    }
    public static CloudQueueMessage GetNextMessage()
    {
        return Queue.PeekMessage() != null ? Queue.GetMessage() : null;
    }
    public static List<CloudQueueMessage> GetAllMessages()
    {
        var count = Queue.RetrieveApproximateMessageCount();
        return Queue.GetMessages(count).ToList();
    }
    public static void DeleteMessage(CloudQueueMessage msg)
    {
        Queue.DeleteMessage(msg);
    }
}

The now gives us a fully functional class that utilizes the Windows Azure SDK. In Part 2 I’ll start building on top of that using the ASP.NET MVC 2 Web Project. Part 2 will be published tomorrow, so stay tuned.

20 thoughts on “Put Stuff in Your Windows Azure Junk Trunk – Repository Base

  1. Curious as to why you invoke Queue.PeekMessage() and, on success, invoke Queue.GetMessage(). With more than one instance, PeekMessage() could find a message which has already been consumed by the time of the second call. The PeekMessage() seems to be superfluous.

  2. nice start.
    nice detailed code snippets.
    a more thorough explanation of what the code is doing and why you’re doing it like you are might be useful for people not that familiar with Windows Azure.

    like why are you RequestingRecycle, why you chose to implement a base class, why you choose to Peek and then Get of your message etc.

  3. The “photos” Blob container was never created, resulted uncaught exception in the JunkTrunk.Storage.Blob.PutBlob procedure.
    I deceided to put “JunkTrunkSetp.CreateContainersQueuesTables();” in the beginning of the HomeController.UpLoadFile() procedure

  4. Your code file and object names don’t match your pictures or instructions. Very Confusing!

    I.e. JunkTrunkInit is StorageTrunkSetup …

  5. Where is RepositoryBase defined? I see it referenced by the Table class, but never mentioned otherwise…

    • The “RepositoryBase” is named such but the in the code example above I’ve called it the “JunkTrunkBase”. Repository is the pattern style, but I named it JunkTrunk just for fun in the code, which has unfortunately caused some confusion.

      • I understand. Just wanted to make sure there wasn’t something I was missing. Thanks for the quick response :)

  6. Could you post the complete solution? That would probably alleviate some of the naming confusion and then no one could complain.

  7. In the Table class in the DeleteMetaDataAndBlob method there is a call to
    Repository.Blob.DeleteBlob(entity.ResourceUri);

    I don’t see where the Repository class is defined.

  8. If you are using the latest versions of the Azure libraries, then the JunkTrunkBase class should use this line instead if you want your code to compile:

    if (!arg.Changes.OfType().Any(change => change.ConfigurationSettingName == configName))

    • For some reason it won’t display it properly, but above there should be angle brackets after OfType and inside it should have RoleEnvironmentConfigurationSettingChange before the parens

  9. Above in the BlobMetaContext class this line:
    CreateQuery(RepositoryBase.TableName);

    Should have After create query and before the open paren.

    • Word Press is filtering out all sections of the code that have an open and closing anglebrackets, which makes it a little tough to read the code snippets.

      Above should be CreateQuery’OpenAngleBracket’SomeType’CloseAngleBracket’(RepositoryBase.TableName);

      • As you’ve noticed, this code no longer compiles because of changes that the .NET Azure Team has made to the SDKs.

        It is best used as a guide if anything. As for the formatting being stripped out, there is a minimal amount I can do about that, as WordPress strips out many of those things to protect against cross-site scripting attacks and such.

        If you’d like to download this complete solution check out the Windows Azure for Developers Code on Github here: https://github.com/Adron/windowsazurefordevelopers

  10. in JunkTrunkBase class
    private static CloudBlobClient BlobClient
    {

    get
    {

    return Account.CreateCloudBlobClient();

    }

    }

    there is an error that Account in not in current context.
    so what to do??
    missing some refference….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s