A Little Bit of Nothing

Ramblings of a madman

Fun With Installing Service Bus for Windows

tl;dr

I recently started a toy project that will involve using Azure Service Bus to communicate between multiple Azure WebJobs and Cloud Services. I’m not a fan of doing development against actual live cloud services so I figured I would install Service Bus 1.1 for Windows Server on my local dev machine to do my development and then deploy a full instance of the application to Azure for cloud testing.

I don’t like developing against live cloud services because this tends to cause collisions among team members (multiple people start posting events for example) and there is a risk of publishing cloud credentials into public repositories.

This post will lay out a bunch of roadblocks I ran into with getting Service Bus up and running on my local dev machine.

My machine:

  • Windows 10 installed with AAD user
  • VS2015
  • Service Bus 1.1

Installation

Installing Service Bus itself is easy. Just use the Microsoft Web Platform Installer. Search for ‘Service Bus’ and install ‘Windows Azure Pack: Service Bus 1.1’.

Search for Service Bus in Platform Installer

Farm Creation

Configuration is where I ran into my first problems. My organization uses Azure AD as a mirror of our main domain AD. Windows 10 gives the option during installation to just login with your AAD account. This is very convenient but leaves your computer in a strange state.

When you use AAD to create your user account your machine isn’t actually connected to your domain. You have a Workgroup machine but are logging in with what feels like a domain account.

Service Bus did not react well to this situation. When creating a new farm using New-SBFarm I would receive the error:

New-SBFarm : Unable to cast object of type ‘System.DirectoryServices.AccountManagement.GroupPrincipal’ to type ‘System.DirectoryServices.AccountManagement.UserPrincipal’.

This error seems to be related to the mismatch of my user (AAD/Domain) to my machine which is tied to a Workgroup. According to the System Requirements, my options were to create a local admin account OR join my machine to a domain and log in with a domain user with admin rights. I opted to add a local user and farm creation worked fine.

I also found that I was not able to grant any sort of permissions to my Service Bus farm and namespace to my AAD user. If you’re on a workgroup you’ll need to use local users for pretty much all interactions with the Bus.

Starting Service Bus

In order to actually get Service Bus up and running the next step is to add your local machine to the farm using New-SBHost. During this step I found that my Service Bus Gateway would never actually start. Digging into the Service Bus logs in the Event Viewer (Applications and Local Services -> Microsoft -> Microsoft-ServiceBus -> Operational) I found an exception that the assembly Microsoft.Cloud.Common.AzureStorage could not be loaded.

Service Bus Gateway service failed to start, retry count 3. Exception message: An error occurred creating the configuration section handler for namespacePolicyDataStoreFactory: Could not load file or assembly ‘Microsoft.Cloud.Common.AzureStorage, Version=2.1.0.0, Culture=neutral, PublicKeyToken=4fe77f22fa8374f3’ or one of its dependencies.

After some searching I stumbled upon this GitHub repo. Apparently, if you install VS2015 you will run into a missing DLL problem because Service Bus for Windows does not ship with this assembly. Normally this isn’t a problem but a change to mscorlib now causes the assembly to be loaded even though the code is not actually used.

The solution is to install a fake version of the dll which contains the types that are being loaded. I’m not terribly excited about registering some random person’s dlls into my system but I disassembled the fake assembly and it truly does created a single empty class to get around a type reference issue. Because the type isn’t actually used the lack of implementation is not an issue.

I downloaded and installed the dll as described in the repo and Service Bus services were up and running.

Hello World?

Service Bus is running, you’ve created a namespace, we should be able to try getting two processes passing messages, right? Well, maybe.

It was during my ‘hello world’ testing that I ran into the most concerning of the issues with setting up a local Service Bus as a dev time replacement for Azure.

I followed the typical Service Bus examples and was met with an exception:

The remote server returned an error: (400) Bad Request. The api-version in the query string is not supported. Either remove it from the Uri or use one of ‘201 2-03,2012-08,2013-04,2013-07’

When you communicate with Service Bus you will typically install the Nuget package WindowsAzure.ServiceBus. Service Bus uses a numbered API version that is verified on connection. It seems Service Bus for Windows 1.1 is pretty out of date compared to what is available in Azure proper. I was forced to downgrade the WindowsAzure.ServiceBus package all the way back to 2.1.4.0 before I could establish a connection with my local Bus.

I’m not sure what the ramifications will be to using such an old version of the Service Bus API when it comes time to deploy my app into Azure. Documentation says the Azure instance supports all the client versions back to the beginning of time so presumably if I’m not trying to use any of the fancy new features I should be okay but time will tell.

I look forward to when Microsoft publishes a new on-prem Service Bus that more closely matches what is actually running in Azure.

Certificate Fun

If you got as far as getting Service Bus running and installed the right client libraries and were met with TLS/SSL security errors you’ve probably run into an issue of certificates.

System.UnauthorizedAccessException was unhandled Message: An unhandled exception of type ‘System.UnauthorizedAccessException’ occurred in Microsoft.ServiceBus.dll Additional information: The token provider was unable to provide a security token while accessing ‘https://localhost:9355/yournamespace/$STS/Windows/’. Token provider returned message: ‘The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.’.

When you add your machine as a Service Bus host with New-SBHost, the configuration script will generate new SSL certificates for your machine. These certificates will be tied to your hostname and if you are on a domain I think they are actually made for you fully qualified domain name. This means that you cannot use other names (like localhost) to connect to your Service Bus. The certificate names won’t match and the trust connection will fail.

Conclusion

Running Service Bus locally for dev is possible but not without configuration annoyances. I haven’t used it enough to know if there are any actual draw backs once you have it working and it comes time to deploy to Azure. Perhaps another post will provide some followup.

Things you’ll need to be aware of for local Service Bus installation:

  • If you are in a workgroup you must use local users, if you are in a domain you must use domain users. If you are on Windows 10 with an AAD user your machine is probably in a workgroup. reference
  • If you have VS 2015, you need to install a fake Microsoft.Cloud.Common.AzureStorage assembly.
  • You must use Nuget package WindowsAzure.ServiceBus 2.1.4.0 or older.
  • You must address your Service Bus connections using your full machine name not a short name or something like localhost

Hopefully this saves someone hours of frustration.

Who Wrote This Sh*t?

Through countless hallways in countless software shops you can hear the words, “Who wrote this shit? This is crap. What an idiot.”

Over the past year I’ve been working on projects that were inherited from other shops and any passerby could hear such phrases coming from my office as well.

Recently a software project that I built went through the heaviest system load it’s ever faced at a relatively large event for our city. The event was crowded, overloading the capacity of the venue and some portions of the system.

Soon, the frustration grew and the Twitter flood of complaints started appearing. People were mostly just generally frustrated/excited/disappointed about the whole situation but I spotted one particular tweet from a local software developer wondering who wrote this shit software.

These words stung and I immediately felt the urge to step up and defend myself, potentially starting a flamewar. Who was this guy to judge my system? He couldn’t possibly know what the causes of the delays were! He couldn’t possibly know the legal, physical and financial constraints the system was under!

What a jerk!

Before I could carefully craft my 140 character pride-filled response I was suddenly struck by the realization that I have asked nearly the same damning question about many other products and applications in the past.

Somehow, in my mind, every piece of software I write is lovingly crafted using the best knowledge and technique that I have at the time. I care about every line that I write and take great pride in my work. On the other hand, that other guy with his “shitty” system is a negligent fool who clearly doesn’t know what he is doing and should leave software development to the people that know better. Everyone is stupid except me and the people that I choose to look up to.

Stop being Arrogant

Although some people are bad at what they do or may simply not care, I have to assume that the majority of people do care and want to do a good job. People generally care about the work that they do and want to do a good job.

I may not like what they’ve built or how they’ve built it but I need to take the default assumption that they did the best that they could with the knowledge and resources that they had. From time to time this assumption will be wrong but I’d rather be wrong assuming the best of people.

Wouldn’t it be better if we spent our time trying to teach and share our experience with those around us rather than cut down our peers to make ourselves feel superior? What if instead of spouting angry ill-informed comments on the internet we took the time to simply tell those people about our experiences and let them know what we think could be better? It’s easy to spout off how frustrated we are and unfair the world is but what could we accomplish if we spent that energy thinking about what could actually be changed to improve the situation?

Thank you, internet stranger. You’ve taught me a valuable lesson and I’ve got a framed print of your tweet to help me never forget.

Make It Happen – Part Deux

You are good enough.

After writing my previous post I kept thinking about the reasons people hold back from going after their goals and aspirations. Then I saw this:

If you care about what you do, its hard not to feel inadequate. @bjeanes

And the very next morning a friend of mine posted this:

Failure can destroy you or make you work harder.

Of course! Fear! Fear of inadequacy, of failure, of embarrassment. Fear can be a paralyzing force for many people.

Well, have I got news for you! You are good enough!

Want to contribute to an open source project? Do it! Your help will be appreciated! The other developers will help guide you and teach you how to improve. And odds are, you will teach everyone else something new too!

Want that new position at work? Do it! Prove yourself!

Will you fail? Probably at some point. But you will learn and you will grow.

I have not failed. I’ve just found 10,000 ways that won’t work.

-Thomas Edison?

There is no secret quality that the people you admire have that you lack. They work hard, they keep trying and they chase their passion. These are all things you can do too!

So get out there! Take one step at a time. Make it happen!

Make It Happen!

Nobody owes you anything.

I could probably leave this post at that but it’s kind of a downer and I prefer to try to inspire people so here it goes…

I’ve been thinking a lot about the idea of entitlement and demanding that others make one’s situation better and I don’t agree with the idea. It is not your employer’s responsibility to give you the tasks that you want. It is not your spouse’s responsibility to make sure you make healthy choices for yourself. You cannot stamp your feet and expect to be able to spend your days in Hawaii and get well paid for it, dammit!

It is YOUR responsibility to make those things happen. Or rather it is your responsibility to make your goals a reality. If you want a change in position at work then make it happen. Prove to the business that the position is in your and their best interest. Prove that you are capable of doing the job. The fact that you have been present in a chair at the company is immaterial to whether or not you should be at a certain position or filling a certain role.

BUT, while it is your responsibility to make those things happen it does not mean that you can’t ask for help. I sincerely hope for your sake that you are in a relationship with your employer/spouse/friend that they want to help you. Your friends genuinely want to see you succeed and will likely bend over backwards to help you if you ask. So push forward, make every attempt to make your goal a reality, and if/when you fall short ask those around you for help.

If you find yourself in a position where everyone around you isn’t supporting your goal there are probably three conclusions to be made: 1) you are surrounding yourself with the wrong people, 2) your idea/goal is not aligned with reality, or 3) people just don’t understand what you’re shooting for. Any of these conclusions should give you pause to rethink your position.

So what about the people around you? Don’t be afraid to help someone around you out. Pick up your neighbor when they are having trouble. Give some kind words to your spouse when they are struggling. Next week that struggling person will probably be you.

It is up to you to make your dreams a reality. Take responsibility but don’t be afraid to ask for help. If you’ve built a good community around you, you might be surprised at who gives you a helping hand.

Adventures With PowerShell: Episode 1 – 64 vs 32 Bit

I have been working to learn/use more Powershell at work and as with any new language/technology the learning process has been fraught with strange gotchas and mysteries. For example, my friend had trouble with Powershell stopping processing because he was clicking wrong!

My most recent learning adventure was that the contents of certain system directories may not be what they appear! I was goofing around with learning how to install Powershell modules into the global location (making the module accessible to all users). The typical advice you get on this matter is simply to drop your module into C:\Windows\System32\WindowsPowerShell\v1.0\Modules and you will be able to Import-Module to your hearts content.

This worked great for me until I tried to load my module with the x86 build of PowerShell. The module could not be found.

Here is the file listing for the global module location from Windows Explorer.

Windows Explorer system32 Listing

And here is the file listing for the global module location as seen by the x64 PowerShell.

x64 Powershell system32 Listing

And here is the file listing for the global module location as seen by the x86 PowerShell.

x86 Powershell system32 Listing

Same file location, different contents. WAT?

After much poking around I finally figured out that when you are running the x86 build of Powershell, the contents of C:\Windows\System32\WindowsPowerShell\v1.0\Modules are actually the contents of C:\Windows\SysWOW64\WindowsPowerShell\v1.0\Modules.

Unbeknownst to me at the time, the developers of PowerShell were actually trying to help me out with this decision. Basically, the remapping of the file system allows you to always use the same pathing without worrying about which version of PowerShell you are running BUT allowing the system to segregate modules that will only work in x64 environments. The tricky bit is that for module authors that want their modules to be available to all versions of PowerShell then they must install the module in both locations. Unfortunately, this behaviour does not seem to be well documented (or my Google-fu needs work).

Good luck with your modules!

Zombies, Run!

I ran the first mission from Zombies, Run! and I absolutely loved it!

If you want to do some running but you’re tired of worrying about pacing, distance, time etc give this game a try. You pop in your headphones, tie on your shoes and don’t get caught! The game guides you through a story and from time to time zombie hordes show up to eat you and force you to sprint. It’s a slick little hook to do sprint/interval training and it’s a blast!

NCrunch Is Awesome!

If you are a C# developer, go try out NCrunch for your continuous testing.

NCrunch will automatically run your test suite as you modify your files. You don’t have to take the time to run your tests as you do your TDD red-green-refactor steps, the tests just keep running.

The visual feedback of what lines are covered by tests and what lines are involved in passing/failing tests give you immediate feedback as you work without a lot of looking around for test results. Mark also made the observation that the black dots remind you to flesh out your testing because they constantly taunt you that you are writing untested code.

There may be some trouble with NCrunch in conjuction with Nugets new automatic retrieval mechanism that was released recently. I haven’t tried it out yet but it sound like there might be something to watch out for there.

Go use it!

Are Checklists Bad?

Checklists. Just the word warms the hearts of safety officers around the world. Who doesn’t love a clean concise list of things to do with those inviting little square boxes just waiting for a pen to delicately mark the items as complete?

Before you go on a trip:

  • Filled the car with gas? Check!
  • Went to the bathroom so we don’t have to stop in 3 minutes? Check!
  • Are you SURE?….. gimme a minute….. Check!
  • Packed some awesome snacks? Check!

That’s great and all but as I see it checklists have three problems when they start popping up beyond your daily chore list:

  1. Nobody cares
  2. Nobody understands
  3. People will game

Nobody Cares

It is not uncommon for people in business to get handed a list of items to do when they work through process X.

In a previous workplace there were issues with the process of checking in code having to do with some brittle system we were using at the time. A checklist of commonly missed tasks was compiled and distributed through to all of the developers. The development processes were also changed so that each developer was required to go through the checklist every time they checked code in.

As the checklist was reviewed with the developers you could almost feel the eyes rolling. The list was not long but it was a little tedious and the developers felt that each item was so obvious and simple that we were almost insulted that someone would present us with a little worksheet that we were supposed to work through every time we submitted code. Ironically, if you asked any one of those developers if they had made a mistake that was listed on the list we would each have to admit that we had.

Although the checklist was thorough and everyone could admit that it would have helped them at some point, every copy of that checklist was dutifully pinned to a cubicle wall and completely ignored. Nobody cared about the list because they felt that they already knew how to do everything listed or they thought it was too tedious to go through the process of performing the tasks listed. The intent of the list was to fix some very real problems but nobody cared.

Nobody Understands

Checklists have their place and are a great way to make sure that you take care of small items that are clearly defined. Airline pilots have all sorts of neat little checklists that keep the planes in the sky. The tricky bit is the clearly defined part.

Let’s return to the pre-vacation list from the previous section:

  • Filled the car with gas? Check!
  • Went to the bathroom so we don’t have to stop in 3 minutes? Check!
  • Are you SURE?….. gimme a minute….. Check!
  • Packed some awesome snacks? Check!

Do you notice anything ambiguous? Dollars to donuts this checklist will give you warm fuzzy feelings as your pull out of the driveway until the crushing moment when little Jimmy asks for his apple slices and peanut butter. You DID pack little Jimmy’s apple slices and peanut butter right?…. RIGHT?!?

How could you miss the apple slices? You had a checklist! And you even went through and checked it!

Checklists tend to turn off the free thinking part of the brain. In fact, that is kind of the point and why things like inspections use them. Regulatory checklists tend to be a list of rules. If the things being inspected follows the rules then it gets a check. Pretty simple stuff.

Checklists like the vacation list above are not like a regulatory checklist. They are created to serve as reminders to someone to do something more complex than what is actually listed. I find that when faced with a checklist people (including myself) tend to stop thinking about the intent of what is on that list and get focused on getting to make the checkmark. I’m sure when you wrote on the list ‘Pack awesome snacks’ you thought that you would go back and consider all the travelers in your packing, but when you got to actually packing you instead pack some walnuts and thought to yourself ‘Walnuts are pretty damn awesome. Checkity-check-check-check!’

The list failed because the intent gets lost in the single minded focus of task completion.

People Will Game

Checklist based performance analysis is one of the best ways I’ve seen to get people to game a system rather than actually doing better at whatever is being analyzed.

This is almost a conscious version of people ignoring the intent of checklist items in the blind draw to get a check mark. The big difference is that now people are extrinsically motivated to become single mindedly focussed on getting things checked off in the fastest, easiest and probably most ineffectual way possible.

All I can say is don’t do this and try to stay out of situations where people do it to you.

So What?

If you work anywhere for long enough you will likely either be handed an exciting new checklist to work through in your job or be tempted to make such a document yourself because people keep making mistakes with some process.

I believe that checklists should be avoided as a way to make sure people are doing the things you want them to do. They will be ignored or gamed most of the time and whoever is giving out the list is likely to be resented for their trouble.

The intent of lists as a process technique makes sense. You have a list of things you want the process to accomplish.

If the list is really simple and tedious then automate it. Computers are really good at doing really tedious stuff so that the humans can eat jellybeans and worry about difficult problems. Computers won’t get bored or distracted half way through and they will probably (hopefully?) do the task much faster.

If the list items are more complicated than a simple script will allow and/or requires human involvement then reformat your process to an activity of some sort rather than a checklist. I have to flesh out this idea but I think in many cases you can rethink what you are trying to do away from a list. Maybe this is just manipulative tap dancing but I think people will probably respond better.

Also, focus in getting people to understand why it is important to do these tasks. If you can get buy in on the why then you will get better involvement in the how. In many cases you will not only get better coverage of your items but you will probably get better quality of work completed. Heck, in explaining the why you might even realize you don’t have to do that task in the first place!