Monday, 31 January 2011

Perforce, Depots and Workspaces

I have started using Perforce Source Control recently and having previously used Visual Sourcesafe and Subversion (with Tortoise in Windows), I thought it would be a pretty straight swap but being new to depots and workspaces is very confusing so I thought I would provide an idiots guide from someone who doesn't understand enough of it to make it confusing (yet!).
  1. The depot is the repository on the server but filtered to only show the section from your CURRENT workspace
  2. The workspace shows ALL of your workspace files from ALL of your workspaces including files/folders that are not present in the depot.
  3. You should create an "ALL" workspace to include the entire depot so that you can always browse to something you haven't got and "Get latest Revision"
For example, suppose you have a depot with folders named A, B, C and you want one workspace for working on A and one for B. You would need to create, initially, an "ALL" workspace including everything. If you click on the depot tab with this workspace selected, you will see A, B and C. On the workspace tab - at this point - you will have nothing because you haven't taken any files.
Imagine you now create and select a new workspace pointing only at A, on the depot tab now, you will only see A and after right-clicking and "Get Latest Revision", your workspace tab will now show A even if you change workspace.
Do the same for B and you will now see A and B in your workspace and B in the depot if that new workspace is selected:

Workspace selectedDepotWorkspace

It is a little confusing but hey, if you have to use it, you have to use it!

Friday, 28 January 2011

I don't like Moodle

I'm sure this will draw the usual comments but I am not a big fan of Moodle. It promises much but has so many weaknesses with basic things that I think the project oversight is not good enough.
For a start, the screen layout is clumsy, both in quality/professionalism but also layout. On some pages there is a left-hand menu, on others is a different left-hand menu that is much more fully styled with icons, some have no menu. Logout is sometimes in the top right, sometimes right at the bottom. Too much use of centre justify means that lists with different e.g. lengths of names, moves the grid and therefore the mouse clicks. It really does look like 100 people have written stuff independently and stuck it together.
Permissions appear to be far too fine-grained which gives a massive list to look through for various permissions, most of which you can only work out by looking at the php source and finding out what is what.
Documentation is astonishingly hard to find for library functions and general usage even though the Moodle site is large and has a lot of information on it. I am not convinced the search is working correctly.
We have been using it for a small distance-learning university and have found many assumptions that cannot be applied to our organisation even though it is a "learning organisation" - not limited to the course structure and the way assignments are handled.
I have actually modified some of the code to suit and added a new assignment type (one of the few areas that looks decent) and here is my next gripe: the code is mostly horrific.
One of the foundations of community projects is well structured and moderated code changes. Things like if...else with two large blocks of mostly similar SQL inline with everything else on a page leading to effectively a single 1000 line long function is not maintainable, quite simply it is asking to break (something I did often but fortunately in new pages I had copied from existing ones).
Look on the other hand at Yii. I know it is a framework rather than an application but actually many of the things are much better thought out. The user interface looks neat and consistent by default, there is a good use of OO for common things like output encoding etc, built in support for translation. It's just a shame that these projects start small and don't consider "big issues" until it is too late to do a fundamental change.

Wednesday, 19 January 2011

Developers - Make your life easier

There are various things that make life easier and where a small investment could yield larger returns (or savings).
Today, we spent a few hours trying to get my login working into a test system. After about an hour, we found out that rather than being defined in an xml file, they were now in a database and we needed a test app to update the database.
When this change was made, all the developer had to do was to delete/move/rename the XML files in our source control and we would have immediately asked the question as to who and why it was changed. 5 minutes saved by the developer cost 2 man-hours of money to the company.
Also, very simple things like having some basic wiki articles on how to do certain things can save the hassle of asking busy people questions and waiting around for answers. This is especially true of anything that might have subtle non-obvious steps like changing permissions or adding test data to databases.
The first time these things happen, learn from them and adjust. They should not happen a second time!

Sunday, 16 January 2011

Pains and Permissions on Web Folders

When you develop web sites in Linux and haven't used it much before, you might find certain things that don't work properly, files don't write or can't be edited or at the other end of the scale, you might have lots of file access open which shouldn't be. Basically, the Linux security mechanism is very simple, everything being treated as a file and it describes permissions for 3 levels of user: the owning User, the Group and (Other) users. Each of these levels can have permission to read and/or write and/or execute the given file. For instance, if you open a console somewhere and type ls -l, you might see something like this:

drwxr-xr-x 6 root root 12288 2011-01-15 12:47 Downloads

The letters at the beginning of the line describe firstly that this is a directory and then the read/write/eXecute permissions for each of Owner, Group and Others. For instance, in this instance the owner can read, write and execute whereas everyone else can only read or execute it. In the case of a directory, execute doesn't mean anything.
This is important because depending on the user and group of the file, you might find that you cannot write something even if you are an admin user. For instance, in the above example, root is the owner and group for this file so if I attempt as user "luke" to write this file, I will not be permitted to do it. If you have unpacked an archive (for instance a PHP website framework) into the /var/www directory, you will probably have used sudo and therefore the files will be root:root by default. The first thing you should do afterwards therefore is change these to be owned by the web server account which is called www-data by default in the Apache web server. We do this by changing ownership (the chown command) like this:

sudo chown -R www-data:www-data ./MyNewSite

The sudo ensures you will have permission to carry out the change and the -R applies the change recursively into the directory you specified. In this case we are assigning the owner to be the USER www-data and the group to be the GROUP www-data (same name, different things). Obviously you could assign different owners or groups than www-data but I find this is easiest in my web directory.
You will now probably face another problem in that even if you are a member of the www-data group (if not, add yourself to it) that the group permission on most files by default is read-only which means as "luke" I still cannot modify these files without using sudo or kdesu to invoke the program. You could change the ownership of the files to "luke" but the easiest thing is to modify the files again but rather than changing ownership, you want to change access modifiers (using chmod) so that the members of the www-data group can also modify these files. We do this:

sudo chmod -R g+w ./MyNewSite

Again, the sudo is to ensure you are allowed to do it and the -R is to recurse directories. Be careful since Linux treats r and R differently and using the wrong r might do something different than expected.

Assuming now that Luke is in the www-data group, I will be able to read AND write any files in my new site but they are still owned by the web server which can access them as required.

Once you have a finished site, it is worth locking things down again and perhaps setting the permissions to a+r-wx which will mean the files are all read only. This will not work if some of the files need to be written by the web server (i.e. log files etc) so be careful what you change. At least make all php files (and any configuration such as connection strings if not php) as readonly.

Friday, 14 January 2011

Hiding MVC Controller Methods

In MVC, all public methods are callable by default in a Controller class. This could obviously not be what you want although I would question why you have public methods that are not callable from the web! Anyway, if you want to hide the method from being treated as an Action, simply mark the method with the [NonAction] attribute (System.Web.Mvc) and a call to the method will return a resource not found error.

Great Quote about Code Maintenance

This is from ASP.Net MVC in Action by Palermo et al and pertains to decisions we make to do something the quick and easy way. The problem with this is described:

"The best practice still stands to avoid putting data access in your presentation layer; any data access concern in a controller action creates technical debt that will put a tax on maintenance for the life of the application"

Well said!

Thursday, 6 January 2011


Something I am a firm believer in is the community learning something and everyone else benefiting. For instance, we don't all have to discover electricity because others have come before us and we all benefit.
In the world of software applications, particularly web applications, the security weaknesses that we all face are very similar and are known - people have done the work for us and we know where vulnerabilities lie so why do we not design these protections in from the start of our systems?
Well there are of course several reasons. Maybe we didn't write the system, maybe it is open source, maybe we are not very well trained or experienced, maybe we mistakenly believe that our site would not be the target of any type of attack.
Whatever the reasons, OWASP has made great progress in defining an industry standard specification which describes fixed levels of web security and provides an objective means to measure both the security of our own applications (meaning that we can prove government compliance for critical web apps) but also a way of measuring the quality of a security tool that we might use to test our code.
The link is here: Application Security Verification Standard Project and as they say, even if you can't follow all of it, anything is better than nothing. Having an incomplete code-review checklist is better than no checklist.
It is worth spending time reading through the materials (owasp also have the free esapi security framework for use in applictions) and if you are a large organisation, probably worth investing in some training. The principles are not difficult but they are also not all obvious and having someone to analyse your processes and point out areas of weakness is usually easier than trying to learn ASVS and analyse your own processes.
Let's hope that one day these kinds of things just become the de-facto standard and that by standardising and learning from each other, we can kiss goodbye to most web-based attacks.