Wednesday, 20 April 2016

KeyVaultClientException: Operation "get" is not allowed

Azure Key Vault is an interesting new service for Azure users that allows you to store keys and/or secrets away from prying eyes. If someone was to break into your server, they would not have access to the keys, although theoretically, they could call into the Key Vault manually to decrypt data.

Some restrictions that are a pain. Secrets, which are pretty much anything you want them to be, are not Hardware Security Module backed so their use is only slightly better than using something like encrypted web configuration files. An attacker could potentially gain access to the Key Vault in the same way they might access your web servers. Another pain is that although you can create keys invisibly and delegate encryption and decryption to a Hardware Security Module, the service only currently supports RSA keys (i.e. public key encrypt/decrypt), which is incredibly slow relative to symmetrical encryption and not something to use for repeated low-latency operations.

Anyway, I wanted to give it a go and followed the tutorial here: which is kind of OK, although like a lot of Microsoft Documentation, the important distilled stuff tends to be spread across lots of documents, even for something simple like the Key Vault, where I want to read headline stuff and be able to drill-down into the details if needed.

I set it all up, added a vault, a key and a secret and then tried to access it from .Net code using the NuGet packages. I got the KeyVaultClientException fire and the message "Operation "get" is not allowed", which sounds kind of obvious except that I had definitely given the permission to my app. Of course, with all the Active Directory requirement, there is another whole level of confusion about what might or might not be broken.

The problem? A simple typo in the secret name (passed as URL) left over from an attempt to use a key, which had "keys" in the path instead of "secrets". I only spotted the problem due to a similar problem on CDNs where you tend to get permission denied errors when attempting to access a non-existent object. I guess it makes sense since the vault is on a public URL and you don't necessarily want people fishing for key names but it is still a pain, especially since the principal did have access to the key vault itself.

The other thing I now need to work out is how to avoid it creating a key vault client and obtaining access tokens for every operation. I can obviously cache the values but how will I know when I need to re-authenticate i.e. when the token expires etc? Just more hassle I suppose.

Wednesday, 13 April 2016

Visual Studio designer files and The name does not exist in the current context

Some things are so unbelievably annoying that I can't believe they are not already fixed. Most of you have probably had the problem where your code behind cannot "see" the controls on the ASPX/ASCX page and you get loads of compiler errors. You might have worked out that the .designer.cs file is not being updated and in some cases you are told to try various trickery to make it all work again.

I have heard about "Convert to web application" (even though I have already been using it as a web application) or even attempting to either delete the designer file or modify the ASPX to force a regenerate but none of these worked for me today.

The only thing I knew, is that I had copied the files from another project into this new one, which I am doing to reduce the massive number of dependent assemblies in the solution. I wanted to keep the namespaces and everything the same but was getting these errors.

Fortunately, I realised in VS2015, which makes it slightly clearer, that although the designer files were visible in Solution Explorer, they were not "included in the project", which makes it understandable that they are not generated and compiled to make it work. Goodness knows why they are not copied as a group, especially since the ASPX and code behind are both copied automatically. Maybe it assumes they will be regenerated but they can't be because it has copied the file but not included it!

Anyway, I had to right-click every single designer file and choose "Include In Project" and surprise, surprise, it all started working again!

Maybe this was also true in previous versions of VS when I have had the problem but Solution Explorer simply didn't make it as clear that the files were present but somehow excluded?

Footnote: It seems that it worked OK when the files themselves were copied across, it was only when they were copied as part of a folder that the designer files were not included into the project!

Wednesday, 6 April 2016

blobExists for Azure PHP SDK 0.4.0 (API 2.2)

After updating my code to use the latest version of Azure's PHP SDK, lots of things changed, including using namespaces but also including the way certain things are done. There used to be a function called blobExists() which was any easy way to, you guessed, check whether a blob exists in a given container.

For some reason, this is not present any more, perhaps because it was previously implemented in the SDK (not the API itself) and hasn't been duplicated but either way, I wanted one and I have written my own version!

Note that I am using it in the Yii framework so there is a yii error log call that you won't need but otherwise it should work. I have also included the namespaces in the code for completeness, although you can instead add a "use" statement to the top of your class.

 * Helper function to find out if the given blob exists on Azure storage
 * @param string $container The container name to look in
 * @param string $blob The blob name to look for
 * @return boolean True if the blob exists
 * @throws \yii\web\ServerErrorHttpException
public static function BlobExists($container, $blob)
        $proxy = \WindowsAzure\Common\ServicesBuilder::getInstance()->createBlobService("DefaultEndpointsProtocol=https;AccountName=accountname;AccountKey=yourkey");
        $options = new \WindowsAzure\Blob\Models\ListBlobsOptions();
        $options->setPrefix(basename($blob));   // Use prefix to limit the number of results
        $result = $proxy->listBlobs($container, $options);
        $blob_array = $result->getBlobs();
        return count($blob_array) == 1;
    catch(\WindowsAzure\Common\ServiceException $e)
        throw new \yii\web\ServerErrorHttpException();

Note that basename() is used since setPrefix() doesn't work with a full filename, only the first part. This will cause problems for files that have the same name but different extensions, in which case, you would need to do some extra checking with the results to do a full match.

There are also other preferred ways to pass the connection data to the ServicesBuilder using environment variables etc. see for details.

Tuesday, 5 April 2016

Using the Azure SDK for PHP - What I learned

I have been using an SDK for Windows Azure in PHP for some time. We run a cloud system and it stores images on Azure blob storage. A web service retrieves these and returns them in API calls. This all worked fine but then I got hit by the dreaded update.

This is what I learned.

Update problems affect all companies I have ever seen who produce APIs but it will affect larger companies like Microsoft much more so.

A normal update to the same API with a new version number is mostly OK because if the worst happens, you simply downgrade and carry on (hopefully). This update however was not normal, it is a polar shift in the SDK, which involves moving the source, changing its name, listing it on Packagist for a Composer install and the whole 9 yards.

I learned firstly that when this happens, many code snippets, blog posts and forum answers are immediately useless but are not obviously useless because they are still called "Azure SDK", "PHP SDK" etc. If the team called it something more specific or gave it a much more specific version, you could tag posts with version numbers (hey Google, how about adding this to your search engine?).

What I also learned is that when this happens, everything can change. Now this would actually be OK if literally everything changed but of course, it doesn't all change. It's still for Azure, it's still called an SDK and it still provides more or less the same functionality but then it gets weird. We now use namespaces (which is nice) but all the classes have now changed. I no longer have some weird long name class to represent a blob but instead a name-spaced class. This would be OK but even the operation has now changed - arguably to a better model but a big change nonetheless. I no longer create a "blob" object and call getBlobData as before. Instead, I create a blob service proxy and call getBlob on the proxy. I can no longer simply echo the result of getBlobData back to the user but have to use fpassthru and pass it $result->getContentStream().

Of course, what I quickly learned is that the documentation, as usual, is pretty much the 80% of the 80/20 rule. In other words, the easy documentation that I can guess is mostly there but the important stuff like how to port my old code to my new is missing (where is the blobExists() function?), how do I get a Blob? A few rough examples but nothing that useful. Why doesn't every method have an example in the docs (even a simple one)? Why is it that if I set the prefix of listBlobs to "something" then it finds "something.png" but if I set the prefix to "something.png", it doesn't despite the documentation that says, "will return all objects whose names start with the prefix". Why don't the automatic properties work properly (to be fair that might be my local version of PHP but why not add it to my rants?)

The one that really made my head explode was getting a very nice API error from Azure when calling listBlobs (or in fact most of the other methods I tried) saying words to the effect that the MAC signature of the request is incorrect (I understand what that means but not why). I am not calculating the MAC (and thank goodness, since in some languages you have to and I would have wasted my time chasing my tail). The problem? I had mistakenly put a space in my queried container name ("uploaded files" instead of "uploadedfiles") and spaces are not permitted in container names, which is fine, although a nicer exception would have been useful!

So that's when I learned that even companies like Microsoft with all their experience and feedback over the years still cannot capture a large enough number of failure cases and provide useful information to the user. Guess how long it would have taken me if the exception was "invalid container name"?

Don't get me started on their new online licensing. I wasted another 4 hours today trying to work out why my colleague couldn't licence his copy of Visual Studio, despite dashing between MSDN, Bizspark,, azure,, 50 similar but slightly different login screens and the subtle but very unhelpful message "Downloaded licence insufficient". Eventually, it all percolated through so hey Microsoft, what about a message saying, "update in progress, try again later"?