Data safety with Teamwork projects

Why worry?

It is a given that your company will suffer a data loss event. There are too many points of failure for this not to happen:

Human failure point
- Somebody mistakenly “modified” or deleted the project or part of it,
- Somebody purposely modified the data (disgruntled employee)
- …
Software failure point
- Bug in the several million lines of program code
- Bug in the several million lines of operating system code,
- A computer virus,
- A rogue script/program that cleans out directories, or configuration files
- …
Computer hardware failure point
- Hard drive crash, memory corruption, etc.
- Power outage/surge
- …
Network failure point
- Screaming network card
- Faulty cable
- Routing failures: corrupted/lost/truncated packets,
- …
Catastrophic failure point
- Theft of computer equipment,
- Lightning hits telephone lines,
- Building burns down
- ...

The significance of the loss will be gauged by the cost needed to replace the data and the time needed to replace that data. Prior to an important deadline, time may not be available. So the loss becomes more poignant. Will the granularity of the backup and the speed to recovery be satisfactory in a time of crisis?

A strategy to address data loss events

Each defense against a data loss will inconvenience the user. It will have both hardware and a media cost. Likewise the time to recovery will be dependent on the data loss event and the preparedness of the data protection system in place. Think of your data safety strategy as a form of risk management. What can you afford to lose? How does your group work? How fast should the data recovery process be? What risks do you want to hedge against most?

The following tools can be used to meet your strategy:

BIM Server Backup Scripting Tools

See the article BIM Server Backup Scripting Tools

Backup via archiving from the client

Description
- Create a PLA from a teamwork client.
Advantages
- Simple and fast
- Contains the library of the project
- Can be continued as a solo project or re-shared
- Reasonable archive
Disadvantages
- Can only be performed manually
- Only contains a snapshot of externally linked files
- No User/Role information preserved
Backup process
- Send a message to all participating users to send their changes. Once the sends are complete, create a PLA.

Backup via saving from the client

Description
- Save a PLN from the teamwork client
Advantages
- Simple and fast
- Smallest representation of the project
- Can be continued as a solo project or re-shared
Disadvantages
- Can only be performed manually
- Only contains links to the library embedded objects will be present in the PLN
- Only contains a snapshot of externally linked files
- No User/Role information preserved
Backup process
- Send a message to all participating users to send their changes. Once the sends are complete, save a PLN.

Backup via exporting project from the BIM Server

Description
- From projects page of BIM Server Manager export a project.
Advantages
- Same advantages as saving a PLN
- Can be moved to a different server without a share process
Disadvantages
- Can only be performed manually
- Only contains links to the library embedded objects will be present in the export
- Only contains snapshot of externally linked files
- No User/Role information preserved
- Can only be re-imported using the BIM Server, remains a teamwork project
Backup process
- Export using BIM Server

Using the BIM Server project backup functionality

Description
- Method to make a snapshot similar to a PLN file of a Teamwork project on set time intervals
Advantages
- Automatic
- Same advantages as saving a PLN
- Can be recovered without re-share
Disadvantages
- Only contains links to the library
- Only contains snapshot of externally linked files
- No User/Role information preserved
- Can only be rolled back to using the BIM Server, remains teamwork project
- Is only valid if the BIM Server is functional. BIM Server configuration corruption will render the backups useless.
Backup process
- Schedule a backup through the teamwork project settings.

Autosave recovery

Description
- Automatic local saving of the project file. In the Work Environment under data safety this can be set up to save the last few steps or save regardless of modifications on a regular time interval.
Advantages
- Automatic
- There is a separate autosave for each Teamwork project as opposed to solo projects where each autosave overwrites the previous autosave.
Disadvantages
- It can happen that a user, who sees the above dialog:
  - notes the "Open selected project"
  - jumps right to "Continue" (-- after all it is a nice large button)
  - And, most importantly, does not register in his/her mind that an autosave file is being opened. This may not be a problem, if the autosave file is intact and correct, but, there is a chance the event that caused ARCHICAD to quit unexpectedly was something that also corrupted the autosave. Blindly sending in an autosave may be more dangerous to the overall project's progress than not having an autosave. Users should be trained to treat new dialogs, like the Project Recovery dialog with attention.
- When an autosave process is running the user will not be able to edit the project (often this is not noticed, since it is scheduled when the user has not used ARCHICAD for a preset time)
- The same event that forced the user to return to an autosave may have also corrupted the autosave file.
Backup process
- Set up in Work Environment under data safety.

BIM Server backup in ARCHICAD 13

See the article ARCHICAD 13 BIM Server Backup

Things to know about data management

Should I test my backups?

Yes! Are your backups readable? Are the projects complete? How fast can you recover from the small to the big problems?Test the steps to recovery from a cold database backup to make sure you haven't left anything out. Make sure you have all the folders documented. Make sure your backup script runs correctly. Make sure you can recover completely on a test machine. For a full test of a cold database backup you will need to modify the DNS of a test machine so that it mimics your BIM Server, while you have your BIM Server off-line. When you modify the DNS Server for this test you should flush your DNS cache. On PC this is ipconfig /flushdns. On the Mac verify that at the terminal prompt when hostname is entered the original BIM Server DNS is returned. During the installation of the BIM Server on the backup test machine the offered machine name must be the original BIM Server name. If the test machine name is not the DNS name of the original BIM server then your DNS server modifications have not been properly recognized and the test recovery will most likely fail.

When I save a project is it guaranteed to be readable later?

No. Of course most of the time a project is readable later. But, to be absolutely sure a project is readable, you need to close it and re-open it.

I would like to save off states of my project, to possibly go back to a creative train of thought that was abandoned, or to later use the idea in a different development. How should I do it?

Two backup methods stand out above the rest:
- Backup via archiving from the client
- Backup via saving from the client
If you save from the client it would be prudent to also save off the libraries used with the project. This way you have a little package that can be later opened as a solo project or re-shared.

Two other tools that you might consider are saving off a project backup or exporting the teamwork project.

Is the BIM Server a good place to leave dormant projects? Should I archive to the BIM Server?

This workflow will not slow down the BIM Server, but it will make your list of projects difficult to navigate, and your cold database backups will take longer to complete. If a problem occurs it will take longer to recover from a cold database backup.Many would favor using the BIM Server only for active projects, but other factors can affect this choice. For instance, long backup times can be shortened by skipping dormant files, that is, only backing up changes. Backing up the server could be optimized, but keep in mind, recovery will depend on the size of all files that need to be replaced. Also you might wish to weigh in the time it took to share and link projects that are on the BIM Server when considering taking them off line.

Am I more likely to have a data loss event near a deadline?

In an unfortunate twist of fate there is a greater likelihood of a data loss event closer to deadlines. Often the anecdote of a special program, designed by Murphy, detects the tension of the user and purposely destroys data at deadlines. Actually it is human nature that the items left to the last moment may be the most complex. An intensity and perhaps recklessness often accompanies the last minute rush to the finish. When more people are updating/publishing, creating virtual scenes, revising madly, the hardware, software and human nerves are pushed to the limits -- a perfect environment for data loss.

Is it necessary to back up TW Data?

TW Data is where local project data including libraries is stored. XP:
- C:Documents and Settings[user]GraphisoftTW Data
MAC:
- /Users/[user]/Library/Application Support/Graphisoft/TW Data
It is unnecessary to backup this folder, since it is mainly a copy of what is on the server. The only difference between it and the server's data from the user's standpoint is the modifications in projects that have not yet been sent. As long as all the modifications that were made on the projects joined on the particular computer have been sent (for all users who use the computer), it is all right to delete this folder. Over time TW Data grows as new libraries are downloaded to the computer and others are no longer needed. Local library management that affects what is stored in TW Data is currently not available. This makes it wise for the CAD manager to occasionally clean TW Data when the size occupied could affect the operation of the machine. To err on the safe side the CAD manager should make sure all of the users who have used the computer have left their Teamwork projects before doing this type of housekeeping.

Can I copy a project backup that was stored to tape/dvd into a ''live'' project backup folder on the BIM Server and use it?

The project backup is found in a folder below Projects in a folder of the form /Name_of_project_- 24B02C07-E699-364D-9AFB-985E7AB8AB56/Backups/2009.12.18 13-08 Project Version – Name_of_project.backup. (the code numbers and GUID will be different!)The *.backup file from backup media can be copied into _any_ live project’s backup folder and then rollbacked to. So be careful, since it is possible to overwrite any project with the backup file.For instance a test backup project could be shared on the BIM Server called “B”.
Move the *.backup file from media to Teamworkserver/Projects/B-619F596E-8073-4887-A534-7A7AC544E628/Backups (note the code numbers are just given as an example). After you copy the backup files to the new directory the server most likely will not recognize them. After the BIM server is stopped and started the files will be present in the project's properties dialog. The computer that the BIM Server is on does not need to be shutdown only the BIM Server needs to be restarted.
With the BIM Server rollback “B” to the *.backup retrieved from backup media –choosing to create a new project
Then sign into "B".

"B" will be the project that was retrieved from backup media. Of course it is also possible to copy the *.backup into the original project folder if so desired. Make sure to test the process before doing it live. Also in this case the BIM Server should be restarted so that the newly added backup is recognized.

I need an emergency way to recover data from my shared project

A commonly overlooked method is to save off a pln from the most up-to-date client in your teamwork group.
1. Go to the “/Users/NAME/Library/Application Support/Graphisoft/TW Data” folder, and find a folder with the project name with numbers in the name, something like “MyHouse342617386”. It should be there, if the current user has ever worked on that project. Back this folder up to prevent any further data loss. Don’t move the folder, just make a copy of it at a safe place
2. Start ARCHICAD,and in ARCHICAD, select Open/Join Project command.
3. In the Open Teamwork Project dialog, at “Choose BIM Server” go for the “All available BIM Servers” option. This may list projects in your “Local Cache”. It will appear with a Red House icon, indicating unsent changes.
4. Open this project, then save it as a PLN, to prevent further data losses.
5. Try to send changes
6. If you can’t send changes, you can recover data from the PLN, either re-sharing the PLN, or re-joining the project and copying data from the PLN to the TW file

My backup software uses port 10000 and it is taken by the BIM Server

The BIM Server might use the 10000 port if it is free when it needs a new port for inter process communication. The use of any specific numbered port is not required, and the BIM Server dynamically searchs for free ports skipping those that are already taken.If the user is using a backup system like beremote (the 10000 port is somewhat of a dead give away), then here is some advice:The beremote server can be changed. See, http://seer.entsupport.symantec.com/docs/255498.htm
Additionally, you can coax the BIM Server to search for ports above 10000 by changing the Model Service Port for the BIM Server to a number above 10000 or delay starting the BIM Server until the software needing 10000 has reserved it.

My X.BIMProject file failed to import

I received the following "Failed to import the project" message:

Copy the BIMProject to the desktop of the computer that is hosting the BIM Server application and try the import from there. The service that runs the import has little to no authority. The BIM Server application is capable of reading and writing to the projects and attachments folders as well as where it was installed. Even if your project is actually on the computer that hosts the BIM Server you might get this message if the folder is a share that has been assigned special permissions to work correctly with network security.