After getting some expert help and someone looking at the source, the problem was that jQuery and Bootstrap were included explicitly by me but then ASP was including another link to jQuery as part of the validation code. This second include, caused the bootstrap extensions to jQuery to be removed and cause the error. I only worked out that it was included for validation since it was in the same area as other code that was obviously related to validation.
I dug into System.Web.dll with ilspy to try and find out what was happening in code and why this additional jQuery is included. The relevant code is shown below:
This shows the method OnPreRender in System.Web.UI.WebControls.BaseValidator which amongst other things calls RegisterUnobtrusiveScript() which in turn registers the jQuery include script. The only way to prevent this script being registered (assuming that you want the validators to be enabled client side) is to change whatever IsUnobtrusive returns to false. IsUnobstrusive looks like this:
And it is clear that the only thing we can change is a property called Page.UnobtrusiveValidationMode, something I had not heard of. Anyway, it was easy enough to set in my (master) page - you could set it in an individual page if required, like this:
Which means you don't want the validation and all dependents to be added in an "unobtrusive way" i.e. in a way that doesn't require the site author to add other dependencies just to make something work out of the box. This simple change meant that the validators didn't add the jquery include and therefore it didn't redfine the $ symbol and break bootstrap.