The Journey to Microservices Part 1 : Top 10 Gotchas (so far…)

tl;dr

  1. Environment Problems
  2. Core Library Development
  3. Team Morale and Momentum
  4. Service Size and Granularity
  5. Resource Allocation
  6. Contract Versioning
  7. API Consistency and Design
  8. Data Modeling
  9. Communication
  10. Logging

8 Months Ago…

The development team at Integrate is a textbook example of a team that hit a breaking point with a monolithic architecture. Our architecture and process made it increasingly difficult to keep up with the intense demands of a rapidly evolving market. It was evident that something needed to change.

We did our due diligence and research and assessed what our options were and ultimately landed on micro-services with a sprinkle of altering our processes.

Fast-Forward 8 Months…

Well…the good news is that it feels like we are somehow right on schedule…(not really, but are you ever in software?). All jokes aside, we are where we need to be and have learned an incredible amount about microservices, processes and ourselves.

Anybody who has put even a few minutes into researching microservices has likely come across some of the following articles…

FYI…I just grabbed those in order from a google search of “microservices”, but I actually did use these and many other resources along the way and so should you.

We did a lot of research and felt we had a pretty good ideas of the common pitfalls and complexities that would be introduced migrating to microservices, and for the most part that research helped us along the way thus far. However, there have been some hiccups along the way that I am surprised did not make it into many (or any in some cases) of these lists.

Our team wanted to put together a brief list of what we felt were our top 10 ‘Gotchas’ thus far in hopes that they might be useful to others researching microservices as we did.

Top 10 ‘Gotchas’

1. Environment Problems

It doesn’t matter if it is your local desktop, staging or production. You will run into issues along the way with migrating from environments that are suitable for monolithic applications to ones that can handle distributed. It may be lost messages on your service bus, or you not being able to get your CI to build your services consistently, but something will happen.

Ramp up your DevOps skill set or hire somebody because you are going to need it. Embrace the reality that the ‘server guys’ now play an integral role in you and your teams successes and failures.

2. Core Library Development

This was a tough one for us to figure out, and nobody seemed to say much on it in other posts so here it is.

Figure out your core library dependency chain and try to structure you team(s) in a way that will reduce chances of any teams having to wait on each other. The time spent waiting for your core service to become stable adds up quickly and is frustrating to everybody.

Give some thought to how you will manage your packages and try to find a suitable package manager. Much like the logging, we did not have to put much time into this as NuGet is the de-facto .NET package manager.

3. Team Morale and Momentum

This was another one that crept up on us. Asking a team of developers to do things differently is never an easy task. Asking them to change the way they communicate and solve problems is an added burden.

In the beginning the many failures will far exceed the few positives that your team will have early on. This can get to a team, and it got to ours. Look for the short wins and don’t get worrisome when any one problem rises, regardless of its impact on momentum.

4. Service Size and Granularity

The time spent discussing whether a service is too big or too small, or should be broken down into smaller services adds up quickly. If you find a golden solution for determining this, then shoot me an email. Otherwise, go with your gut and be open to the idea that your gut is occasionally wrong.

But really…don’t spend too much time discussing one way or the other. The true solution will expose itself with time.

5. Resource Allocation

With any big decisions impacting your business, you should consider whether or not you have the resources to see it through. Resources may be personnel, hardware or even training on new technologies. We had this in mind from the beginning, but it still seemed to catch us off guard

Depending on whether you introduce microservices in small chunks or all at once might influence the amount of resources needed. To avoid this causing any catastrophic problems for your team, make sure you are on the same page as your organization in regards to the resources you are expecting to need.

6. Contract Versioning

I have to tread lightly so I don’t start a troll battle between versioning camps…

This is partially related to #2 on the list because of the amount of time we spent during our core service development introducing breaking changes until we saw them stabilize.

Choose a versioning strategy for your contracts and services that will position your team to minimize the amount of downtime caused by frequent breaking changes early in microservice development. For us, the answer so far has been Semantic Versioning, but there are many methods that may make more sense for your team.

Additionally, consider who will be consuming your contracts and services and set expectations up front…If you have no intentions of third parties consuming your services, then don’t trouble yourselves.

7. API Consistency and Design

The consistency of your APIs can significantly impact the time it takes for you to successfully get microservices to production. In order to avoid any single API Ambassador having to manage the consistency and standards of your API design, put aside time early on to discuss how you can keep your API consistently through frequently updated documentation and defined standards.

Take into consideration internal and external consumer flow. We have operated under the assumption our API would include both internal and external API Gateways. This is important to us because if we designed our APIs around a single consumer flow, then we would hit a breaking point in the very near future.

There is an excellent article written by Netflix on accounting for multiple consumer flows and how it impacts your API design, Embracing differences inside Netflix.

Also,determine whether you are using PUT or PATCH for partial updates because we spent way too long in this.

8. Data Modeling

Eventually, you will need to determine how your data will be maintained between microservices. Hopefully, this happens sooner than later.

How will your consumers retrieve data that is assembled from multiple services? Will each of your services maintain a copy of the data to avoid expensive round trips?

Whether you plan on taking a Domain driven approach or you have other methods, do your best but to try to pick a strategy and stick with it. You will find out whether or not it is right soon enough.

We are still working to figure out what works best for us us but we haven’t ran into any significant issues on either side of the spectrum that would convince us we are right or wrong thus far.

9. Communication

If you haven’t read up on Conways Law then I would suggest reading it. If you don’t believe it… then start believing it.

Chances are, migrating to microservices will require at least some modifications to your team structure, if not significant. With these changes come communication bottlenecks and breakdowns. Many of the communication channels that your team(s) may have utilized before may be obsolete in a distributed environment. More importantly, you may want to place constraints on certain channels to facilitate autonomous thinking within teams. Companies like Amazon has paved the way for these added constraints and have realized significant benefits in service design.

A popular rant by Steve Yegge is scattered over the internet that reflects how serious a company such as Amazon enforces constraints on communication between its service teams.

As a team,we have only recently began to see the reasoning behind these methods, but are learning quickly that the level of autonomy that is needed to scale our teams in a distributed environment is significant and setting these constraints can help facilitate this evolution.

10. Logging

Debugging distributed applications is much more complex than a non-distributed. The IDE and tool support for debugging is horrid at best; and you will find yourself on parts of the internet that you do not want to be trying to fix issues. (Seriously, anything past the 3rd page on Google is just creepy…)

Logging can help bring order to the distributed chaos that are microservices. There are many centralized logging solutions out there and they hardly differ from one another with the exception of support and pricing. Get used to the idea that logging will play a crucial role in your team diagnosing issues.

Very little time was spent by our team choosing how we should implement logging. We defined our needs upfront and ended up going the open source route with Apache Log4Net and Graylog2.

What Color Theme Is Most Popular In Visual Studio?

I recently got into a friendly argument with co-workers regarding what is the better color theme preference for Visual Studio. In my experience, some development teams have an affinity to a certain color scheme while other teams may be different. Without looking too much into this, I thought I would leave it up to a small sample of the community to determine what is the preferred color theme.

To view the real time results for this survey, visit Preferred Visual Studio Color Theme Result Page

Create your free online surveys with SurveyMonkey , the world’s leading questionnaire tool.

Most Anticipated Feature Implementation for C# 6.0 Survey Results

I wanted to post the results of the What is the most anticipated feature implementation for C# 6.0? survey that was recently posted.

Over 200 people contributed to the survey, however since I do not have licensed access to the underlying survey provider, the results I can analyze and report on are limited to 100.

Most Anticipated Features

  1. Null Propogation – 54%
  2. Auto Property Initializers – 13%
  3. Using Static Members – 6%

Least Anticipated Features

  1. Parameterless Struct Constructors – 0%
  2. Exception Filters – 0%
  3. Fast Integer Casts – 0%
  4. Checked and Unchecked Blocks – 0%

What is the most anticipated feature implementation for C# 6.0?


I recently come across the Language Feature Implementation Status for Dev14 table on CodePlex for the .NET Compiler Platform (“Roslyn”). I discussed briefly with some of my co-workers about some of the features that would and wouldn’t be implemented and thought it might be interesting to see what some of the most anticipated features are for C# 6.0.

My plan is to let this survey bake in the Microsoft community for a few days and then post an analysis on the results.

To view the real time results for this survey, visit C# 6.0 Feature Implementation Results Page

Visit the .NET Compiler Platform (“Roslyn”) CodePlex page for more information.

It would be great to get a large sample for this survey so make sure to share this!

Create your free online surveys with SurveyMonkey , the world’s leading questionnaire tool.

Clarifying Passing Argument ‘By-Value’ or ‘By-Reference’ in C#

As a C# developer, I have been approached countless times with the following question…

Do methods in C# receive their arguments by value or by reference?

If you are a C# developer (or really any developer) and have not been approached with this question, you are in the minority.

I felt I have always been able to answer this question correctly, but have not truly understood my reasoning. This lead me to seek clarification on how these languages handle passing arguments to their functions.

After researching different articles/books/blogs that attempt to explain this, I was able to gain a better understanding.

Are method arguments in C# passed by value or by reference?

First, we want to make sure that we give the textbook answer that will get you through most interviews and friendly discussions.

The following explanation was taken from MSDN – Passing Parameters in C#

In C#, arguments can be passed to parameters either by value or by reference…

To clarify, this says that argument CAN be passed by valur or by reference, but it goes on to explain that by default, arguments are passed by value. In order to pass by reference, you must use the ref  keyword.

 What does this actually mean?

Assuming we are strictly talking about the inherent pass by value behavior that C# is defaulted to (no ref keyword like .NET), this means the following:

  • When an object is passed as an argument to a method, the method receives a reference to the object. However the object reference itself is passed by value.
  • Properties of the original object CAN be modified, but the object reference itself CANNOT.

What does this look like in code?

Using the two implementations of ChangeAge(…) given below, we can visualize what passing by value and passing by reference looks like in C#.

The first implementation does not include the ref keyword for the Person argument, therefore is defaulted to pass by value.

The second implementation specifies the ref keyword for the Person argument.

If we have a simple console application such as the following:

We end up with the following console output. Note that only the second invocation of ChangeAge(…) modifies the Age property of the Person object.

What is the explanation of this behavior?

As we mentioned earlier, in C#, you can pass an argument by value and also by reference.

The first implementation of ChangeAge(…) does not specify the ref keyword, therefore does not pass a value by reference, but by value. The confusion here is because it still seems we are passing a reference to the Person object we have created. What we essentially have is a copy of the reference and not the actual reference to the Person object. This is why the modification fails, because we are attempting to replace the object itself before modifying the value.

If we want to replace the object iself, we must utilize the ref keyword and pass the Person object as an explicit reference.

Hopefully this wil clarify some of the differences of passing arguments by value or reference.

Enforcing Method Parameter Conventions Using Roslyn Diagnostics With C# Part 2: Code Fix Provider

Introduction and Recap

In my last post, Enforcing Method Parameter Conventions Using Roslyn Diagnostics With C# Part 1:Diagnostics , I showed how you can use the .NET Compiler Platform (Roslyn) to enforce conventions in your code base by implementing code diagnostics.

More specifically, we looked at enforcing method parameter conventions for a given user defined type (or interface). If you want additional insight on the sample convention, please refer to the previous post. To recap, our convention has the following properties.

  • Any public service method in the internal service layer must specify a parameter type of IInternalUser.
  • The IInternalUser parameter must be specified first in the method signature.
  • Only implemented members of service interfaces can be public. Everything else must be private.
Along with these properties are several assumptions that are mentioned in the previous article in order to keep this example short and to the point. The result of the last post left us with a diagnostic that gave us an error or warning if this convention was not followed. An example of this is shown in the screenshot below.

This was a very basic task and provided realistic benefits based on our convention. However, we want to also be able to provide suggested code fixes when this occurs. For this example we want to rearrange the parameters in the method signature to satisfy our convention. This is where the ICodeFixProvider helps.

Using ICodeFixProvider, we can specify instructions for replacing the existing syntax tree with an updated tree which includes our transformed fixes. This is not quite as trivial as implementing our DiagnosticAnalyzer.

The Implementation

We already have implemented our code diagnostic, so we just our code fix. Assuming you have followed the directions correctly in the first article, you should still have your template CodeFixProvider.cs in your project. It should look the same as the following screenshot.

The template code provides a fix that will make declarations all upper case, but we do not care about this.  Our goal is to take the improperly ordered parameter list and replace it with one that satisfies our convention.

Step 1: Implement GetFixesAsync

We need to rewrite the logic for GetFixesAsync to find the method declaration(s) that our implemented diagnostic was created for and return a code action for each that will invoke the fix that we implement. This leaves us with a fairly straight forward implementation for GetFixesAsync  as the logic for transforming the syntax tree will be housed in our own FixIInternalUserOrdinal which we will implement in the next step.

Go ahead and replace the GetFixesAsync with the following implementation.

I attempted to comment anything that was not straight forward and will not spend time going over each line of this implementation. If you would are having trouble understanding any of this, I would recommend spending a few minutes reviewing Getting-Started – Syntax Transformation (C#).

It is important to understand that we are grabbing the method declaration from the diagnostic that was provided, and then passing it into our FixIInternalUserOrdinal. 

Step 2: Implement FixIInternalUserOrdinal

Unlike GetFixesAsync which has no logic for transforming the syntax tree, FixIInternalUserOrdinal  is entirely responsible for this transformation and contains the logic for reordering the parameters in our method signature to satisfy our convention.

If you have not done it already, you can delete MakeUppercaseAsync  and add the following code.

Again, I am not going to spend time explaining each of these lines of code as I have done my best to explain it through the use of comments.

Instead, I will give a brief summary of the steps I took to reorder the parameters in the method signature. It is important to note that syntax trees are immutable. This means that even if the API exposes methods such as Replace(..) and WithXXX(…), it is actually creating a new object. The summary below uses terminology loosely but you should still keep this in mind.

  1. Grab a reference to the original parameter list node from the method declaration.
  2. Filter the list of parameter nodes to those with which use IdentifierNameSyntax so we can search for any tokens named “IInternalUser”.
  3. Iterate over each parameter in this filtered list until we find one that has a token matching “IInternalUser”.
  4. Create a new parameter list node and add the IInternalUser parameter node as the first item in the parameter node list.
  5. Iterate over the rest of the parameters and add to the new parameter list if not the IInternalUser  parameter node.
  6. Grab old syntax tree root and replace the old parameter list node with the new one that was just created with the correct parameter list.
  7. Return the new document with the transformed syntax tree.

Result!

You should now be able to test and even debug your new diagnostic by hittingControl + F5. This should spawn a new Visual Studio Instance in which you can use our previous project example that should use your diagnostic as an extension.

You should see the following diagnostic feedback.

Now that you have implemented a code fix, you should see a light bulb overlay appear in the left margin.

If you click on this, you should see our code fix Reorder Parameters.


The final result should look like the following in which we no longer see any diagnostic feedback because our convention is now satisfied!

Syntax Tree Visualizer

When you download the Roslyn preview, you can also utilize a useful tool within Visual Studio 2013, The Syntax Tree Visualizer. This proved to be very helpful as I was manipulating the syntax trees using transforms.  As an example, I have provided a before and after screenshot of our test code following the code fix. If you look closely, you will see the highlighted parameter (which represents our IInternalUser parameter) change positions under the method declaration syntax.

The screenshot below shows the syntax tree before the code fix.

Here is the same syntax tree after our code fix has been executed.

We have implemented a diagnostic and code fix to satisfy our convention. In our next article, I will probably try a more complex example.

Feedback Appreciated

I understand that there is several if not dozens of different ways that this example could have been accomplished. If you seem something I could have done more efficiently, or something is done incorrectly, let me know! This is all new to me and I am enjoying getting to know the Roslyn API. I appreciate any feedback that can be given on the subject matter. Additionally, if you have any ideas on conventions that you might have on your development team that you think would be a good example, let me know as well!

Enforcing Method Parameter Conventions Using Roslyn Diagnostics With C# Part 1: Diagnostics

Introduction

With all the hype coming off Microsoft’s BUILD 2014 Developer Conference and making The Roslyn Project open source, I wanted to take some time and familiarize myself with some of the concepts that were covered in Future of C# talk given by Mads Torgersen and Dustin Campbell.

Among several other announcements, they gave a demo on how to implement a ‘Diagnostic with Code Fix’.  In the example, they focus on implementing a diagnostic for conditionals used without braces and also provide the one click fix, similar to ReSharper.

When Roslyn was first announced, I did not quite see the benefit or need to implement custom code diagnostics and fixes. Having since then been a part of a larger enterprise team with a large code base and a strict set of standards and conventions, I now see a niche opportunity to experiment with this functionality.

Aside from the documented source code for Roslyn, it is still fairly early and not many examples are available. Additionally, I try to avoid spending too much of my free development time browsing source code (although I probably shouldn’t). Thus, I wanted to find something that I could easily implement and would provide a significant impact across our code base.

In this article, I am not going to show how to configure Roslyn or Visual Studio 2013. If you need help with this, then you can go use the two following links as resources.

The Convention

I wont get into too much detail on the conventions we use and for privacy reasons, I will modify our conventions for this example.

Our code base follows a service-oriented approach. Additionally, we break down our services into business,internal and external services.

For this example, I will assume that we have the following convention on our internal service layer.

  • Any public service method in the internal service layer must specify a parameter of type IInternalUser.
  • The IInternalUser parameter must be specified first in the method signature.
  • Only implemented members of service interfaces can be public. Everything else must be private.
Invalid Implementations:
Valid Implementations:
For this example, we are not going to worry about checking whether the method is an implementation of the service interface.
Pretty straight forward!

 The Implementation

For this article, we will only focus on the Diagnostic Analyzer and leave the Code Fix for Part 2.

Assuming that you have properly followed the directions that were provided for setting up Roslyn, you should have a Visual Studio instance loaded with a project containing the following template code files.

  • DiagnositcAnalyzer.cs
  • CodeFixProvider.cs
You may have some other files included, but they are not relevant for this article. We are going to only focus on DiagnosticAnalyzer.cs. If you open DiagnosticAnalyzer.cs, you should see template code such as the following:

It is nice that they provided us with template code, but we are going to move right along.

As mentioned earlier, we want to provide feedback in the form of a warning or error, for any method signature that contains a parameter of type IInternalUser that is not listed as the first parameter. This is of course a very basic example.

Step 1: Implement AnalyzeSymbol

We need to rewrite the logic for analyzing symbols from the templated code to what we want.

Replace the template AnalyzeSymbol  method with the following code:

This code simply checks to see if any method parameters are of type IInternalUser and checks the position if necessary. If it finds a parameter of type IInternalUser and it is not first in the list of parameters, it creates a diagnostic and invokes the addDiagnostic callback.

Step 2: Modify SymbolKindsOfInterest Auto Property Getter

In order to notify of the type of symbol(s) we want to analyze in AnalyzeSymbol , we need to modify the getter for the SymbolKindsOfInterest property to return an ImmutableArray<SymbolKind> representing SymbolKind.Method.  This is very simple and simply requires replacing the SymbolKindsOfInterest property with the following:

Step 3: Modify DiagnosticAnalyzer Meta Data Constants

The final step is the simplest and is only required to give proper context to the diagnostic you have just created. For this particular example, I simply want the feedback to the user that it is required that the IInternalUser parameter when present must be first in the parameter list. This can be done by changing the Description  and MessageFormat constants to the following:

You can of course provide a format string for the MessageFormat constant, but I see no need for this example.

Result!

You should now be able to test and even debug your new diagnostic by hitting Control + F5. This should spawn a new Visual Studio Instance in which you can create a new project that should use your diagnostic as an extension.

So why would somebody want to enforce conventions or standards on method signatures? You may not have realized it, but most of the common libraries we use also follow some type of pattern or standard for method signatures. This is why you can typically expect parameters to stay the same as you iterate over method overloads.

Some additional examples of why you might enforce method signatures and parameters are shown in the following list:

  • Ensure that one character parameter names are not used.
  • Ensure that parameter names are camel cased (or whatever).
  • Ensure that you are or aren’t using certain parameter types.
  • Utilize conventions for method names and return types (i.e. returns void is prefixed with “Is” or “Check”).
  • Restrict the maximum amount of parameters used in any method signature.

Stay tuned for the next article where we will implement a CodeFixProvider that will automatically rearrange the parameters so that they correctly match our convention!