There are a number of components you probably want to back up first.  Start with your svn repository and your TAC database.  If you have local projects, then zip those directories as a form of backup.  Of course, do all of this when the applications are not running.

For TAC, start  your old TAC server, and use the Export Parameters button on the Configuration page.  Now shut down the old TAC and start  the new TAC instance.  In the dblogin dialog window on the new tac server you will see an Import Parameters button.  Use that to import your old settings.  This is documented in 5.5 the Installation Guide.  the procedure there says to uninstall the old TAC webapp from your Tomcat instance.  I suggest that you install to a new Tomcat instance so that you have a fallback path.  Just be sure to stop the old tomcat instance as well as the old TAC so that there are no port conflicts.  Also, if you have set CATALINA_HOME be sure to change that if necessary.

You will also have to upgrade your projects in Studio.  Be aware that you may not be able to move these back to the old version once you upgraded them.  Hence the reason for the backup.

For Studio projects that are shared, everything is in SVN already.  So just install the new Studio version, configure the connection to the svn server, and  the projects show up.  For local projects use "Import Existing Projects as Local" in the new Studio workspace.

Then rebuild any bundles with the new Studio and publish them to the Archiva server. 

For runtime instances I am not aware of any tools for upgrading or copying a configuration from one server to another.  You might be tempted to just copy the container/etc folder from your old instance to the new.  That will certainly preserve your configurations, but it will overwrite any new or changed settings added by the upgrade.  In general, you could more narrowly control your specific changes with scripts.  There are sample scripts for server configuration in the script directory of the container.  You can use these as a template if you like. 

Just make sure that none of your custom settings are impacted by upgrades.  This is seldom the case, since most modifications are very basic, e.g. changing port numbers.

Of course, after you install the new runtime instances, and assuming they are running on the same ports, then you can have either the old runtime instance or the new running at any one point in time (on the same machine). 

Assuming that is your approach, you can then simply redeploy all your old bundles/features from the ESB conductor in TAC.  Then, Group or Sort by the server in TAC.  Update the entries in ESB conductor to point to the latest published version, and redeploy them.

A different approach is to simply name the new servers something different and run them on a new port or a new machine.  I prefer this approach because it makes rolling things back much safer.  Just add the new Servers in the Servers section of TAC.  Group by Server in ESB Conductor.  Duplicate all the target features.  Edit the duplicated features to point to the new server instance and use the new bundles/features.  Deploy and test them. Once they are running delete the old ones at your leisure.

Note that although it is a bit tedious to go through all the port settings, it is quite possible to run multiple runtime instances on the same physical server.  I recommend this when upgrading because it preserves a very direct fallback capability by completely isolating new and old instances.