Friday, 9 April 2021

System.MissingMemberException: Public member 'GetSecretAsync' on type 'KeyVaultClient' not found

This could also happen for any "Public member on type not found" combinations.

This can happen any time you have a late-bound method call in VB .Net that does not exist in the object. In my case, the short answer is that you cannot late-bind extension methods!

In VB .Net, if you specify concrete types for your variables and/or function return types, then the compiler can early-bind the calls and work out exactly what is pointing to what. If you have mistyped, Visual Studio can show you where you've gone wrong. If the compiler doesn't know what the type is, it cannot determine if the method even exists until runtime, at which point it might fall over with a MissingMemberException.

Dim earlyBound As String = MyFunction()

earlyBound.BadMethod()       ' Compiler error, String does not contain the member BadMethod

Public MyFunction() As String

   Return "something"

End Function

 

' Late bound example

Dim myVariable = MyFunction()

myVariable.DoSomething()       ' Late-bound, the compiler doesn't know what type myVariable is

Public Function MyFunction()

   Return "something"

End Function

So what about extension methods? 

An extension method is a static method in a static class that the compiler interprets on your behalf. An extension method always has a first "this" parameter of the target type and the compiler effectively converts this:

myString.ExtensionMethod()

into this

ExtensionClass.ExtensionMethod(myString)

That is all fine except for one important detail. The extension methods must reside in a different class to the class they are targetting because they must be in a static class.

Why is this an issue? Image we have a StringExtensions class and one method called TruncateLongString(string As String, length As Integer) and imagine we have the following late-bound code:

Dim myVariable = MyFunction()         ' MyFunction returns a string but the compiler does not know this

myVariable.TruncateLongString(50)    ' Compiler will allow this and will make a late-bind attempt

At runtime, when the code is actually executed, what actually happens under the covers would be something like this:

Dim myVariable = MyFunction()

myStringVariable = CType(myVariable, String)  ' Since I now know it is a String

myStringVariable.TruncateLongString(50) ' MissingMemberException: String does not contain TruncateLongString(), which is correct, it doesn't!

So how do we avoid this? Use types for your Functions and ideally your variables.


A council hit by ransomware attack AGAIN

 A ransomware attack has cost Redcar and Cleveland council an estimate £10M to resolve and an article today debates how much of that the government should have provided to help them.

As a software professional, I find these stories infuriating in that a large, exposed, public body like a council, which is paying, probably at least £200K per year in salaries for their IT staff are still not protecting adequately against and more importantly having a disaster recovery plan for such a common attack vector.

In one sense, I don't think the government should give them any money because it is their own fault. However, of course, we are talking about elected people who are incompetent enough to allow this all to happen on their watch but it is the public's money that is being wasted. There are no real sanctions in this country for people who fail basic competency in Information Security, at worst they will lost their job but the local taxpayers are still on the hook for the lost money.

Ransomware is a surprisingly common and effective attack. An attacker attempts to get some malware to run inside a network, if they can, the malware encrypts whatever it can and shows a message to the victim asking for an amount of money (usually reasonable but not insignificant - the attacker wants a good payback but if they charge too much, they are more likely to not get paid). If the attacker is paid, they will probably release the files (it would be bad for business if people didn't believe they would get their data back), but of course, there would then be a danger of a further attack since you would not know for sure whether the attacker would a) completely leave your system and b) give you help to patch your systems to avoid further problems.

As I said before, these are very well known attack vectors and organisations should be learning from existing attacks because the attack is the same and should be dealth with in the same way.

Firstly, there are two main ways to prevent the attack or at least reduce its impact. The first, and hardest, is attempting to prevent malware from entering the system. In most cases, this is almost impossible since everyone uses email, virus scanners don't always pick up new malware but in most cases, an up-to-date scanner; user education; locking out USB ports so people cannot introduce malware from personal USB keys; and also a very strict policy on people who attempt to circumvent these policies in the same way you would treat someone who messed around with the systems on a plane or messed around with banking software.

You cannot ever have 100% assurance that this will work so you also need to secondly reduce the attack surface and this is about network/file permissions and also about data segregation. If you have, for example, an application that processes council tax payments, it should absolutely not be visible across the entire council network but should be isolated by something as simple as subnets so that one system cannot directly address another. Of course, this is sometimes tricky because of people who might have access to multiple systems but that is why the CISO is paid a lot of money. You also need to consider the ability to hop via different people's machines across the whole network via e.g. a council-wide shared network folder, again, something that is probably not needed in most cases.

The second crucial issue is having a policy so that people recognise when ransomeware is happening and you have a kill switch that can be operated to disable certain systems before they might have been affected. It will involve a combination of powering down network switches and even pulling power cords out of crucial servers. If you do not have any means to do this, it is more than likely that one of your staff will sit there watching a malware screen, go for a coffee and then try and call the IT department by which time it is too late. You can get special software that monitors for this, again, an up to date virus scanner can sometimes spot ransomware.

Thirdly, you aboslutely must have a disaster recover policy and process. The policy dictates how much money and time you will invest in setting it up, for example, how often you will take backups and how they are secured so that backups are not encrypted with the rest of the system (it has happened!). The process is literally a document that the IT team follow if you suffer from ransomware e.g. take down the network, have a network of people to go round and tell people what is happening, have a process for checking PCs before they are rebooted, working out how the ransomware arrived; getting backups restored, communicating with your customers: all very basic Information Security materials.

There is always an irony when an organisation won't spend e.g. £100K setting this all up and end up paying £10M in reparations.

Please people, start taking this seriously!

Tuesday, 6 April 2021

Remote working for Developers...or not?

 Just before the covid lockdown last year, we were preparing our first serious experiment into remote working for our small development team. At the time, we had 2 seniors, 2 juniors and me but as a UK-based company we suffered in many ways from the lack of suitable developers in the job market and were looking at recruiting remotely.

It was a risk, of course, since none of us had done it before. We started with a simple test where one of our senior devs worked from home 3 days per week and then came into the office for the other two. This worked OK and we realised that it certainly could work to get people working remotely full-time.

Then lockdown happened and we were forced into it anyway but fortunately the pre-work of setting up RDP and VPN and also the experience of working remotely for that short period made the transition relatively painless, including for the rest of the company who were not planning to work remotely but who could now use our new remote working systems!

I highly recommend the book "Remote: Office not required" by Jason Fried and David Heinemeier Hansson, the founders of Basecamp. They were in the rather unique situation where their two original founders of Basecamp were already remote, one in the US and the other in Europe. They had to work out how to work across timezones, and in this case, an extreme timezone gap but then decided to build the company in this way, learning what is important to make remote working work well.

There are of course, many reasons why employees like remote working, and in the case of the book, completely flexible hours, such as family commitments, leisure time (e.g. surfing) that might work better at some times of day than others and also the very genuine advantage of being able to move house and keep the same job.

The definite main stick-out point from the book (other than trying to keep people within +/- 2 hours of timezone to avoid extreme challenges), is that you must be able to measure productivity if you are to ensure that people aren't doing what many managers are concerned about, which is not very much, and to ensure they are adding appropriate value to the company for their salary.

One of the ironies of office-based working is that we assume if people are tapping away at their keyboards for 8+ hours per day, we assume they are working hard and providing value. If we think about this, however, it is absurd to equate attendance and even effort with output or value. In other words, whether we work from the office or remotely, management need to have some fair and objective ways of measuring performance - not to 2 decimal points, but certainly in a ballpark.

This is where it gets tricky and why some of the larger tech companies have talked a lot about remote working yet proving by their actions that they still want people back in the office to provide that false sense of security/productivity. The truth is that measuring output is hard, at least partially subjective and it doesn't scale.

How can you measure how much value that a developer adds to the team? In extremes, you can certainly spot the very effective from the non-effective. You can spot very poor quality from very high quality but most people are in the middle and do not stand out. 

Counting tasks is not great. Juniors are likely to work on the easier tasks that don't involve risk. Seniors might really struggle, not because they are not working and not because they are not skilled but because some problems are hard and these might be infrequent enough to make it hard to average across individuals.

I have not found any specific way of achieving this but there are some principles that can help setup a system that is fair and ideally, might be automatable.

  1. Compare people with their peers, not with an arbitrary industry standard
  2. Measure performance over time
  3. Have a think about what you need to measure and how you can do this
  4. Use the team to be accountable to each other

Firstly, every organisation is different and has a different speed/quality tradeoff. The amount of functionality you would expect to achieve in a financial application compared to something more noddy is very different. Industry averages will not help here so we need to compare our senior devs with other senior devs, mid-level with mid-level and juniors with juniors. We are not looking for complete parity but we should be mindful of salaries and salary reviews when we measure people. It wouldn't be fair for the most productive dev to be paid less than the least productive one.

Secondly, it is probably obvious that things change over time. New starters will have less familiarity with the systems and processes, some weeks you get given a nasty task that is really hard and might be pushing the boundaries of what is possible within your framework and you are not inclined to add 100s of lines of code for something that sounds like it should be easier. Over time, we would expect new people's work rate to increase and occasional bumps to be ironed out. I do actually track tasks counts in sprints as one metric to help me measure performance, this at least helps me spot outliers like people working too fast (quality issues) or not fast enough (cannot focus, lacks ability etc).

The third part is the hard part. What can you actually measure to know how effective people are? I think there are 3 things we want in the "ideal" developer. a) People write high quality code b) quickly c) without creating bugs.

High quality code is quite subjective but there are certain things that mark good code from average or bad. High quality unit tests (where useful); good use of patterns; lack of code smells and general housekeeping (tidy code). This is hard to measure consistently but one place is in a code review. You can count average comments/file (more files would expect more comments), ideally removing question comments that might be like "have you considered X", "Yes". This would take several months to take allowance of the different ability of reviewers, different motiviations to pick up everything rather than just important things and the fact that some of our code might be better than others. We might ideally want to distinguish a simple error that anyone could make from a howler that should have been spotted but this would generally require the developers to count these for you and fill in a form to record it. You can then tie these stats specifically to goal setting and even potentially the ability to slow people down in order to improve their scores - there needs to be some "cost" to people producing poor quality code as a motivation to do it right.

Doing things quickly is the sign of a proficient developer. I remember a Contractor at a previous company who was paid (a lot) to write an encryption library to centralise the methods we were using for hashing and encryption into a single maintainable place. He took a month and it wasn't finished. After he left, I rewrote it from scratch in 2 weeks and finished it. Although crude, if you set the principle that no tasks should take longer than 2 hours, then counting tasks is a way to count speed.

Speed is nice but quality is essential. In the modern web, there are so many ways to get attacked for different reasons, and there is a large risk that the organisation takes when deploying any application. What happens if an attacker works out how to delete everything? Even with a backup, you lose credibility, you have some data loss while restoring the backup not to mention that you still have to track and fix the bug that allowed this to happen. As agility increases, the change of getting bugs in production is increased and it is essential therefore that the developers a) understand their ability to cause critical bugs b) know how to know if a bug has occurred c) have the ability to jump on it and fix it quickly and d) know that there is a consequence to what they have done both to the company and potentially to their job!

If bugs are found in production, they need to be traced back to the cause and a root-cause made to ask whether it could or should have been avoided. If the code is horrible and the bug not-surprising, then why wasn't the code refactored before being used? If a developer hasn't followed the correct process and caused a bug that shouldn't have happened, do they need mandatory training or a conversation about whether this is the right job for them? Most of us like to play happy families but any manager will have to deal with people who will not or cannot do what is required of them in a role and you definitely want to find this out sooner rather than later.

Lastly, if you don't want to be seen as a Manager who incites fear in the team, you need to use the wider team to monitor and to help each other. If the sprint is a team effort, then a single person not pulling their weight is probably visible to other members of the team who will want to speak out since the individual affects the perception of the entire team. You are not looking for blame culture but if someone is not producing anything, they need to be called out. If they are not producing high quality, it needs calling out. If the team does this as well as the Manager, then you create a team culture which is easier to understand for new candidates and easier to see if they are not a good fit. The team can then provide input on how to improve, training required etc.

So, in conclusion, remote working requires performance monitoring but so does office-based work. Secondly, measuring performance is not easy but without it, you are just sticking a finger in the air, which means people who want to game it, will game it.


Friday, 26 March 2021

The challenge of Node docker images

TL:DR; Not all node images are equal. Just because one doesn't work doesn't mean another won't.

I want to deploy a container running the highcharts node export server, which should be super easy and is exactly what Docker was designed for.

I found an existing github project that already has this functionality. It runs perfectly but:

1) The node version is very old (7.7)

2) It is based on a large Linux container at 800MB

So I thought it would be easy to update the version and also use something like Alpine or Buster-slim to make it better. Note that I have already installed the server onto a new installation of node latest version on Windows server and it does run as-designed.

First problem

I tried using a really new image (15.12-buster). When I did this, it built OK but I got some sort of node error when calling the server: "TypeError: worker.restart(...).work is not a function".

I tried reducing the version one-by-one assuming that the export server was based on some older functionality (even though I had installed it fine on node 15 recently). No dice. There are hundreds of tags for node images and not all versions include all versions of the linux kernels (obviously) but eventually I got back to using the original 800MB 7.7 bare image, just to prove I hadn't gone mad.

Second problem

I then tried the Alpine image. This really is the daddy because it is really small (100MB) and suited to a really small node app. Or not.

Alpine uses a different c library to Ubuntu/Debian (for size presumably), but this doesn't work with Phantomjs, which although derided in some circles is used by the export server that I need to install.

I tried a hack to download and install it directly before running npm install but for some reason, npm install didn't find the installed version and tried to download and install it. This lead to my...

Third problem

The node base images have everything installed as root, including the global node modules, but NOT /home/node which is the location you are supposed to install your apps at.

This was really confusing because running npm install caused permission errors even though I was running as root (afaik) and installing a global module. Since the permission error is logged without showing which user was denied, it was hard to know if I had to fiddle with the node user or whether it was another issue that looked like a permissions issue: for example a missing directory that needed creating.

Fourth problem

I then thought I would try each different image to find one that worked and was the smallest. I attempted to use the slim/buster-slim libraries and then got an error with bzip2. It seems this is not installed on buster or buster slim so I added an apt-get update and install but then had the same permission problem with it trying to install.

Fifth problem

When I tried the wheezy image (using the old version of node), running npm install segfaulted. An issue I found was along the lines of "never mind, we won't be supporting this any more", so no real help there.

Eventually

I had to accept that for now, I had to use the original old version at 800MB and just deal with it! I could probably spend some more time and try and build up my own node image manually but I'm not sure if that is time worth spending since it is only to save a couple of GB.

Conclusion

Although I have reasonable experience with Docker and Linux, this has used up a couple of hours for something that should have been much easier than this. You might need to build your own images from alpine or you might have to spend a lot of time trying different images, it is up to you!

Friday, 19 March 2021

Developers are still not writing effective CVs

 I am interested in CVs because I think as an engineer, I should be able to evaluate a good CV analytically and if I am writing my own, I should easily understand what is important and what is not. However, having read 5 CVs for Development Managers recently, I realise this is still not standard practice across the industry, which is disappointing. It is also worrying that recruiters do not coach their candidates to write effective CVs.

If I only get 1 CV, I might fight through the messy CV, ask some questions of the recruiter and then decide if I am still interested. If I get 10 or more CVs, it is not worth my time. Each CV can easily take 5 minutes to read, plus an email plus replies which might or might not include further conversations etc. all adds up to a lot of time. The truth is there are probably lots of people who are perfect for the role and the organisation but who quite simply cannot adequately describe themselves in a CV.

Also, in this day and age, I have only seen one candidate who had an online/web site for a CV.

So what are people getting wrong?

Too Long

Not too common but still some people who send a CV with 6 pages. Guess what? I Don't care what you did in IT in the 1990s it is 99.9% irrelevant. If you have been out of school for 10 years or more, I don't need every single job you did since then unless a) they are relevant and b) they are a 1-liner. What I do care about is the most recent 5 years and to a lesser extend the 5 years before that.

Not describing your personality

Not everybody will fit into every role or organisation due to culture: the culture of the organisation and the culture of the individual. You can save a lot of time by describing yourself in soft-terms, not just in technical terms. 
 
Are you the kind of person that is very careful and methodical to ensure quality is 100%? Great, you will fit in really well in some places and not in others. Are you the kind of person who can be trusted with free reign to produce an effective development environment or are you more likely a trusty steed who works hard and long and gets a lot of work out of the door as long as someone creates the work list for you?

Obviously, you should avoid negatives like "I sometimes struggle to get motivated" but also in yor introduction, it is useful to include some traits. Don't put too many in, otherwise you will sound unreasonable or proud.
  • Driven
  • Careful
  • Visionary
  • Efficiency-driven
  • Motivated
  • Innovative
  • Team enabler
  • High-level picture
  • Detail-oriented

Describing the Company and not your input

This is very common. Some describes a role like "Senior Developer, Acme Ltd. Worked on systems that are used by garages to handle inventory and other insurance applications...." In other words, you are describing what Acme does and not what you did. It might be nice to know that Acme works in insurance or the automatic industry for some context but I want to know what you did specifically e.g. "Worked on a team of 5 in all aspects of the design and development of the system. Took a lead in web application security and managed the training of junior developers"

Not picking out your headlines

Now this is also common but might actually be because you have never done anything of note! Hopefully not but make sure you include this to pick out what you are good at. e.g.
  • Migrated our app from web forms to dotnet core
  • Managed our offshore development team
  • Introduced a new deployment process to reduce times down from 2 hours to 5 minutes
  • Introduced Scrum to the development team

These are more than just individual topics, they speak to motivation, authority (decision-making), skillsets and experiences that not everyone will have. A good question for candidates is "what did you ever do that you were not asked to". You'd be surprised how many people would say, "nothing". 

Not accurately conveying your technical skillset

This comes in a number of forms. Some people simply record the "toolset" used in a particular role. Some list a set of tools, languages or processes at the top and some people don't put much at all except in the introduction, "experienced .Net developer".

Another form is when people record their skillset in terms of years.

None of this really conveys the skill level you think you are to me as an employer.

Your company used SQL Server? Irrelevant. You have experience with SQL Server? Not enough information. You have used SQL Server for 15 years? Again, says nothing. 

Instead, it is important that you give yourself a reasonably-based assessment, either using 1-5 or perhaps beginner, intermediate, advanced. It doesn't matter that these are not completely objective but if you claim to be an expert, it gives me the opportunity to ask you some advanced questions about that topic to see whether a) you are indeed that skilled and b) you are a good judge of your own skills.

It doesn't take long in an interview for me to ask people (as I have) "What makes you an expert in HTML then?". You can't bullshit the answer to someone who understands HTML (without being an expert).

Weasel words

These are words or phrases that are added to make something sound impressive even though they don't actually say anything. They are more common for people who have come from the corporate world.

e.g.

  • Was responsible for ensuring stakeholder needs were met
  • Ensured a balanced set of priorities for resourcing projects

These might seem fair to start with but I would expect every single person in my organisation to ensure that stakeholders needs are met and that priorities are balanced, that is not a headline for a CV. A better way of saying something that might actually be genuine would be.

  • Managing a very large backlog of work that was produced by 20+ stakeholders for our product and effectively communicating how priorities can be agreed so that the most valuable work is done first.

Conclusion

Most people seem to find it hard to understand that the CV is a way of describing you to someone who has never met you and knows nothing about you. Knowing this as well as the fact they are looking for a developer and not a new best mate, you should be able to work out what sort of information is relevant to your CV and what adds nothing.

Secondly, people don't always realise that their CV is a competitive weapon to make them look better than the other candidates. It needs to have things that wow the reader, that make you look better than other developers with the same experience, this is by following the guidelines above and highlighting achievements as well as things you did that you were not asked to do but you wanted to in order to make your small slice of the world better.

Think like this and improve your CVs!

Monday, 15 March 2021

Skoda stop-start not working

 I bought an Octavia estate last year just before lock down and was pleased to own the newest and nicest car I ever had. it had all the mod cons including an automatic stop-start system, which I am sure you all know about.

After a while I had a number of issues with it, and the car generally. I took it to the approved dealer and they said there were no fault codes.

The problems included:

occasional high rpm before gear change (it's automatic)

start stop would not cut the engine due to high energy usage, even though nothing was obviously running

sometimes the engine would really struggle to start

one time the display showed a stop start error but mostly it was just strange behaviour.

Well it seems like it was related to a low battery, probably caused by lots of short journies during lockdown. The last time I took it to the garage, it wouldn't start again! I bought a small charger from Amazon for 30 quid and as soon as I connected it, it showed a very low battery so I left it to fully charge.

Next time I went out, the car started easily, the stop start cut within a few minutes when I stopped despite the air con running. it all looks like it might be good.

However, for an unreliable component like a battery, why on earth wouldn't the car detect the low voltage and show the error on the dashboard? I have already paid the garage 60 quid originally to try and fault find when a charger might have been the only problem!

Tuesday, 2 March 2021

Optimise Docker builds by tweaking the Dockerfile

 After having played around with Docker a bit more, I have learned some more things about optimising Dockerfiles to make builds more efficient.

As mentioned previously, Docker has a cacheable layer system built-in, which makes sense, since you are not just building e.g. a web app but in a sense building the entire OS, additional modules/plugins/runtimes and then your app as well. The build could easily take 10 minutes+ if you run the whole lot, so you clearly don't want to do that just because you changed one file.

So you need to understand how Docker builds and caches layers so that you can work out how to make the optimal balance between build times and flexibility.

Each time docker reaches a command like copy or run, it creates a new layer. This layer takes place in an "intermediate" container, which is deleted by default after it is used since the resulting later, which is really just a directory diff, is then cached locally. When you build the dockerfile the next time, if Docker detects that any source images or copied files have not changed, it simply takes the cached layer and applies that directly. This is the difference between running e.g. npm install, which might take 30 seconds or more and roughly the 1 second it takes to detect the lack of changes and take the layer from the cache.

What this means in crude terms is to put the least changing content as early in the dockerfile as possible. The Source image (which is often an sdk image for build purposes) is needed to start with so you can't really move that regardless of how often it changes but you can decide whether to pass --pull to docker build in order to get a later version if it exists, otherwise it will use an already downloaded image if that tag exists.

On top of that, it is best to add any global tools you might need for your build but which are not likely to change. For example, I install a specific version of NPM and then install gulp as a global package. Since npm install and npm run dev are both quite slow, I do these next because we don't change the source content as much as we change our code files (you might need this the other way round).

Even when you then get to the point where you might be using a different image for the runtime, you should then layer the least changing content earlier on for the same reason that the cached layering can be used.

Another slowness will come if you are manually copying a lot of files one after the other. To use the caching, each command (e.g. COPY) will create a new layer via a temporary container, which is quite slow and expensive. In most cases, especially if running a combination of related commands, you should combine them into a multiple line command so they are run as a single layer. You need to be careful with copying files because firstly, the copy globbing pattern takes some getting used to (make sure you test it is doing what you expect!) but secondly, if you combine too much, the entire step has to be repeated if any part of it changes. Otherwise, maybe just order the copy commands from least likely to change first to most likely last!

And lastly, make sure you keep testing and optimising. Just changing one cs file in my build and running Docker build is taking 10 minutes. I'm not sure why it is so slow, even copying the output from the build into the runtime seems to take 30 seconds, which doesn't sound right. I do have a Dockerfile that builds in debug for tests (which are not currently running) and then in release for deployment, so 2 builds is obviously quite slow but it is still too slow for my liking!