Friday, 22 February 2019

A new Team City Windows Agent using Agent Push

Team City, like many CI servers, supports multiple agents to offload builds. These can be a mix of Linux and Windows (and Mac?) and can have separate capabilities but the one thing that promises to make things really easy is the idea of pushing everything you need to a new agent rather than spending ages configuring it.

Of course, the truth is not quite as simple and the Team City documentation is generally overwhelming and messy and missing some vital steps!

Fortunately, some of us believe in briefer guides so here it is: adding a new windows build server as a Team City Agent.

  1. Get yourself a windows server - I used Server 2016.
  2. If you are planning to start up and destroy agents on-demand, you might want to invest in static IP addresses so that the firewall config etc. on the TC server (if relevant) doesn't need changing each time you spin the agent back up and it gets another IP address.
  3. Log in to the windows server and make sure you can access your TC server URL i.e. firewall open, https setup on Team City. This will be tested by the agent install
  4. Install the JRE minimum 1.8 on the Agent. This will change some time after April 2019 when the JRE is being ditched for packaged Java classes.
  5. Ensure that port 445 for SMB is open. You can lock this down to the TC Server IP for extra security
  6. Optionally take the windows update hit straight away!
  7. Go into TC server, Agents, Agent Push and click Install Agent
  8. Enter the destination IP/URL and login creds. You do not need to enter different ones for the service, it will run fine as Local Service.
  9. You should see a log of activity as the server attempts to install the build agent on the remote machine including any tools configured under the server->Administration->Tools
  10. Once completed, you can optionally create a profile so you can easily run the install again later.
  11. It should appear quickly under the Disconnected agents list and will show "will be upgraded" for a few minutes, at which point, it will appear in the Connected list.
  12. Note that if your new agent doesn't have various tooling installed (Node, MSBuild etc.) then it won't meet the requirements of certain builds and therefore the agent will not be used for those builds. You can see this by clicking the agent and looking under Compatible Configurations. You can unfold the links and see what dependencies are required/missing under the right-hand side.
  13. To install extra tooling, you need to login to your windows server, install the relevant tools and then restart the build agent service, which should take a couple of minutes, after which the server should show the new capabilities.
Remember that if this is your first off-server build agent, any build steps that write to the local disk will write to the agent's local disk whereas you should ideally have an "upload via sftp" or whatever to correctly send artifacts from remote agents back to your deployment server.

Wednesday, 20 February 2019

FakeItEasy unit test error - System.NotImplementedException : The method or operation is not implemented.

So I was trying to set up a fake and got an error pointing towards an unimplemented ToString() method in my fake db context.

The reason? I was attempting to mock a call on an object that was not a fake! I should have created my context as A.Fake() whereas I was using the built-in EF FakeSmartDbContext(), which of course is not a fake object!

Presumably, FakeItEasy was trying to calling ToString to generate a useful error but for reasons unknown, FakeSmartDbContext has ToString coded with a NotImplementedException!

Replace FakeSmartDbContext with A.Fake() and we're all good!

MSB3268: The primary reference could not be resolved - Team City

...because it has an indirect dependency on the framework assembly "System.Data.Common, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" which could not be resolved in the currently targeted framework. ".NETFramework,Version=v4.6.1". To resolve this problem, either remove the reference "something.dll" or retarget your application to a framework version which contains "System.Data.Common, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".

Well that is a mouthful and an error that occured on our Team City build server but NOT on our local machines. It is always a bit of a pig to try and find out what is different between the two but first to find out about this missing reference.

1) The error was raised from a .Net web site that was pulling in something.dll from another project in the same library. Unfortunately, this library referenced a few other solution projects so to track down the culprit, I had to find out which of the dependencies was the first to fail, since that was probably the culprit.

2) The dependency was something like Site -> A ->B ->C and C was copied OK but A and B were not so I decided that it was most likely that B was the problem.

3) Opening B from my local machine in ILSpy, I used the "Load Dependencies" feature to find these missing libraries (System.Data.Common and a few others) and lo and behold, we were referencing DnsClient 1.2 from NuGet which has a dependency on a net standard library System.Buffers >= 4.4.0 We had installed the DnsClient so it had automatically pulled System.Buffers 4.4.0 and it all built OK locally.

4) It turns out that System.Buffers is actually a netstandard library and version 4.4.0 doesn't directly support Net Framework 4.6.1. Of course you can have netstandard support but that involves upgrading the library (and all its descendents) to net 4.7.2 (recommended) and also to change the package management method from the net framework style to the net core style. A lot of work and redeployment we didn't want.

5) Looking at System.Buffers 4.5.0, we noticed that it actually mentioned Net framework 4.5 with no further dependencies, which I think is poor behaviour from MS since they have made a major change in a minor version.

6) Installing a dependent library can be tricky because I was concerned that if I just did an in-place upgrade of System.Buffers in nuget, it might not correctly reload the target framework as a reinstall would do but I couldn't reinstall it since DnsClient relied on it. What I did was uninstall both, install System.Buffers first and then install DnsClient. Not sure if I needed to do that but it worked.

Then push back up to the build server and it was all fine!

It is not obvious what was happening and I still don't know why Visual Studio was behaving itself whereas msbuild wasn't but Nuget publishers need to be much more careful about what versions they support and nuget should really refuse to install netstandard dependencies on .Net 4.6.1 rather than pretend its OK when it isn't.

I think another way is to add the netstandard nuget package to the library but that potentially brings in various additional things that we don't actually need.

Tuesday, 12 February 2019

Converting .Net Web Site to Web Application

So you have a .Net website and want to convert it to a web application? Firstly, why might you want to do it?

  1. More strict compilation
  2. Better reference handling - for example, it will pull in dependencies of dependencies automatically
  3. More deployment options than just "xcopy"
  4. More clever incremental build i.e. won't always build everything every time.
Fortunately, the process is not too difficult, but there are some gotchas along the way. Assuming you are a fairly good developer, here are the steps in short format:

  1.  Build and run the current website to ensure there is nothing fundamentally broken
  2. Create a new empty web forms application of the correct language for the source web site (if you get this wrong, you will get some very weird compiler errors!).
  3. Delete any files and folders that get created with it and prune any nuget or references that you don't want.
  4. You can pre-emptively add any nuget and project references that you know you need or add them later when your compilation fails. You only need to add packages that are needed directly, it will automatically pull in dependencies of dependencies so don't just add everything that is currently in the old bin folder!
  5. Copy the old App_Code folder into the new project, with its contents (which you can do in Solution Explorer)
  6. Select all the code files and ensure their build action is set to "Compile", you can do this in one go with multi-select.
  7. Click on the web application project and then select Project->Convert to web application in the Visual Studio menu. This will rename App_Code to Old_App_Code, which it is supposed to, so don't rename it back.
  8. Perform your first build and fix any build errors. Depending on how loose your web site was written, there could be missing Import statements and problems with anything that is not defined correctly. Note, you can add global imports like Microsoft.VisualBasic into the project settings references section to avoid adding these into each page. Go sparingly though or you will have conflicts.
  9. Copy the remaining content over and then select all of your ASPXs etc. and choose Project->Convert to web application again, which should add a designer file and correctly order the files with the code-behind and designer hidden away. I had to do a lot of this manually but it might be because I missed a step.
  10. You will notice if a page does not have a designer file if you attempt to reference a page control from the code behind and it fails. Simply select the offending page and run Project->Convert to web application again.

Common Errors

Common errors include:
  • Having the wrong build action on a file
  • Not having fully-qualified namespaces in the Inherits attribute of an aspx page
  • Having duplicate class names which would have been namespaced differently in the web site and might now conflict. 
  • "Convert to web application" makes some changes to class names and namespaces, which might cause unexpected errors.
  • If your new project does not globally include namespaces that were included before, files might fail the build
  • If your new namespace is different, you might get unusual conflicts with types that worked before. Types in code files use relative namespace paths whereas Imports/using always use global (absolute) namespace paths. For example type A.B.ClassName referenced inside a class that is inside A.C either needs an Imports/using for A.B or a fully-qualified namespace of B.ClassName (note the missing A). This might have worked fine in the old namespace. 
  • Any times you have used tools like Resharper while migrating might have made incorrect changes - for example pointed ASPX pages to the wrong code file.
Note that like all Web Forms projects, the ASPX page errors will not fail the build by default so make sure you open each one and check for errors to be fixed before running your application.