Case Study:
NASA’s Standard
Autonomous File Server (SAFS) Project
NASA has chosen SoftLink’s technology to transport satellite images in its Standard Autonomous File Server (SAFS) project. The SAFS team has implemented FEST technology to enable the “hands-off” automation of SAFS.
What is SAFS?
The purpose of SAFS is to provide automated management of large data files without interfering with the assets involved in the acquisition of the data. It operates as a standalone solution, monitoring itself, and providing an automated level of fail-over processing to enhance reliability. The SAFS system provides a quicker, more reliable file distribution for customers of near real-time data than has been realized by previous methods.
Initially, the SAFS system was installed at NASA’s Ground Network sites for distributed acquisition of satellite data in support of the QuickSCAT and ADEOS II missions. It was also installed at NASA’s Goddard Space Flight Center to provide centralized customer data distribution. Since the summer of 1998, these SAFS systems have been successfully involved in the integration, testing, and launch support events at the QuickSCAT Mission Operations Centers at the University of Colorado, NASA’s ground stations, and with QuickSCAT customers. SAFS is proving to be both timely and dependable, and should add significant value to the scientific data community.
What role does FEST play in the SAFS Project?
In the SAFS projects, FEST and FASTCopy are responsible for the management, automation and transfer of files ranging from 3 MB to 370 MB in size. The files are distributed to multiple customers with file latencies from 1 to 11 hours. Files acquired by the system are processed and distributed automatically to various ground stations and customers. The SAFS team has deployed FEST technology and configured “Hot” directories to which the acquired files are transferred. From these points FEST filters the files, activates processes and then starts FASTCopy jobs to complete and verify the file transport.
File Acquisition Automation is accomplished through the use of FEST “Hot” directories to detect when files arrive on the SAFS system. These directories have a monitoring process with a time selectable interval for checking file arrivals. It is possible that a file transferred from a TMP might take longer than the checking interval for the “Hot” directory. When this happens, only a partial file would be detected and processed. To avoid this problem, the “Hot” directory can be set to react to the arrival of a flag file, which contains only the names of the files to process. The TMP sends the flag file last, after sending the data files to the “Hot” directory. When the monitoring process detects the flag file, it triggers the moving of the named data files to a temporary directory for pre-processing by a job file and the deletion of the flag file. The flag file is not needed if FASTCopy is being used for file transfer, as it can be set to use a temporary directory to completely capture the file before automatically moving it to the “Hot” directory for processing, or it can automatically start the processing in a non-hot directory. The flag file can also be used to process files in groups, such as a data file with an accompanying metadata file.
GROUND STATION SAFS:
Each of these SAFS systems uses FASTCopy to automatically push these files to the central SAFS, where they are made available to each project’s customers.
More insightful information can be found at:
www.wff.nasa.gov/~code584/projects/ssemancik/pdf/iafpaper.pdf
For further details please contact Sales at: sales@softlink.com