This has become something I have had to do a few times now and rather than keeping scratching my head to remember what to do, I thought I would describe the process of creating an Ubuntu LAMP stack on AWS which is never unnecessarily exposed to the web before it is hardened.
Firstly, create a new security group. It should have only the ssh port enabled on 22 and be locked down only to your public IP address, it can also have the rule for your eventual ssh port added, either open to the whole world (if like me you move around a lot) or your company/home public IP(s) if you usually only access it from one place.
Once your security group is setup, create your new Ubuntu instance (the same instructions will be pretty much true of most Linux distros). And then choose to create a new key-pair or use an existing one if you have one - I prefer to create multiple keys to reduce the chance of one being stolen/obtained and used to access all systems, despite the slight additional hassle of this. Obviously choose to use your new security group for this new instance. You could alternatively have a security group you use for all new instances and then a second group to use once the instance has been hardened.
Run up the new instance and connect to it. If you are connecting from Linux, you will need to add the private key for the instance to your ssh keys (there are plenty of guides but you probably already know how to do this anyway). If you are using Windows, you can use Putty but you first need to use Puttygen to convert the downloaded PEM into a Putty PPK since Putty has its own key format for some very annoying reason. To do this, run puttygen and File-Load Private Key. Change the file filter to *.* and find your key.pem and select it, putty gen will automatically import it and then you have to choose Save Private Key and add an optional passphrase - this would mean that even if someone else had access to the key e.g. access to a shared pc, they would still need the passphrase to connect.
Once you have generated the ppk file for the key (and added an optional passphrase to it), you can create a Putty session that uses the public dns for your instance and port 22. You then need to set the key under Connection-SSH-Auth in the private key file box. Note, the aws instances do not allow password access by default - which is good - so if you do not specify the key (or an incorrect one) when you try and connect, you will get a console error which says Permission Denied (publickey). If your firewall rule is using the wrong ip address, you will get a dialog saying Permission Denied when you attempt to connect from Putty.
Once you are in, the first thing I always do is to run sudo apt-get update and sudo apt-get upgrade to get any package updates before I do anything else.
The next thing to do is to sudo nano /etc/ssh/sshd_config which will change the configuration for the ssh daemon. When you edit this, the first thing you want to do is move the default port from 22 to, let's say, 8989 which is not standard and so is unlikely to be be attacked. There are a couple of other things to do like disable PasswordAuthentication (set it to no) and ensure that PermitRootLogin is also set to no since you will not be able to login as root anyway (the private key uses the pre-created user "ubuntu") and this adds another defence against an attacker who could otherwise do anything that root can do on the box. Once you have edited and saved it, sudo service ssh restart and then ctrl-d to log off. Now the port will have changed and your current putty session will no longer be able to connect. If you have not yet done so, you can remove your port 22 from your security group, change to a new security group and/or add a rule for your new port optionally locked down to specific hosts.
You can now edit your Putty config and point it at the new port (and save your settings). When you connect again, you will get another warning about trusting the servers key which you can go past and you should be into your box again.