HF Radar National Data Management Development

Eric Terrill  (SIO)

Link to NOAA Strategic Plan: NOAA's Mission Goal 1: Protect, Restore, and Manage the Use of Coastal and Ocean Resources through Ecosystem-based Management; NOAA Goal 2: Understand Climate Variability and Change to Enhance Society's Ability to Plan and Respond; NOAA Goal 3: Serve Society's Needs for Weather and Water Information; NOAA Goal 4: Support the Nation's Commerce with Information for Safe, Efficient, and Environmentally Sound Transportation; NOAA Goal 5: Provide Critical Support for NOAA's Mission

RESEARCH OBJECTIVES AND SPECIFIC PLANS TO ACHIEVE THEM

Local, state, regional, and federal discussions directed towards the establishment of an Integrated Ocean Observing System (IOOS) continue to emphasize a desire for the installation, development, and operation of a network of surface current mapping systems for use by a broad range of end users. Central to the operational success of a large-scale network is a scalable data management, storage, access, and delivery system. The objectives for this project are to continue the research, development, and implementation of a prototype data management system for ocean surface current information derived from HF radar. The architecture of the HF-Radar Network lends itself well to a distributed real-time network and serves as a model for networking sensors on a national level. This joint university-NOAA partnership is focused on defining and meeting the expressed needs for an IT architecture supporting a national network of surface current mapping data systems. Research and implementation efforts met expectations during this reporting period. Total vectors are now processed on three grid resolutions for both coasts accommodating for different range resolutions of data. Four data portals have been configured and deployed at different radar operation sites around the country, including the National Data Buoy Center (NDBC). This project has also supported the research and analysis of the Multiple Signal Classification (MUSIC) algorithm for application to quality control of HFR-derived surface currents.

RESEARCH ACCOMPLISHMENTS

This past year, significant progress has been made to design and code the data transport mechanism. From a broad perspective, the HF-Radar Network architecture is comprised of two building blocks, portals and nodes, each representing Antelope (Antelope, a product of Boulder Real Time Technologies (BRTT, www.brtt.com), is an integrated collection of programs for real-time data collection of environmental monitoring information.) enabled computers with distinct roles. Portals serve as ‘point of entry’ machines by acquiring and serving radial data from any number of HF Radar sites. Nodes serve as data concentrators by collecting radial data from any number of portals (or nodes). This design minimizes data requests through occasionally unstable network connections to individual sites by serving data through portals while maintaining a high degree of network flexibility through selective data collection at nodes.

Design elements for each system depend on executables within the Antelope framework. Portals essentially utilize two executables called hfradar2orb and orbserver for acquiring and serving radial data respectively (Fig. 1). The module hfradar2orb was designed specifically for HF Radar file formats, which fall into two broad categories of range-bin and LatLonUV (LLUV). Based on community input and necessity for streamlining process, hfradar2orb incorporates a perl module (codartools) to convert range-bin format files to LLUV and verify file contents. Node communications are comprised mainly of three executables, orb2orb, orbserver, and orbhfradar2db. Data transfer within the network is accomplished via orb2orb and orbserver communications. The executable orbhfradar2db then unpacketizes data files and stores links to them in the Datascope database. Efforts during FY06 have been directed towards deploying additional portals and for developing appropriate metadata for radial files.

Simulations of the MUSIC algorithm were conducted to develop QA/QC metrics for flagging questionable surface current retrievals. Using a MATLAB simulation, detailed analysis of the compact antenna array patterns and the internal signal processing within the MUSIC algorithm was performed on varying simulated input current models. Examining the statistics of the MUSIC results leads to the definition of a goodness-of-fit quality metric for the output radial current velocities and bearings produced by the HF Radar system. To achieve this, theory behind the MUSIC direction finding algorithm, describing its Direction of Arrival (DOA) metric, was developed. Simulations were then conducted and statistics collected on the DOA metrics. The magnitudes of these metrics are directly related to the quality of the bearings produced by the MUSIC algorithm. Each result is presented along side its associated metric to provide an estimation of quality. This research provides HF Radar users with a practical quality metric for the radial current velocities and their associated bearings produced by the HF Radar system.

Fig. 1 Growth curves of HF radar operators and the number of reporting radar sites whose data are managed by the prototype data management system developed in this program. Over 900,000 data records presently reside within the system