FD2020 Analysis

From Stu2
Jump to navigation Jump to search

Field Day provided an interesting opportunity to gather the data from the Reverse Beacon Network and analyze our CW station performance. Woodbridge Wireless (W4IY) used two K3S radios. One stayed on 40M the entire time and the other split duty between 20m and 40m. Our antennas consisted of a single AB-577 50' portable tower to support three antennas; 80M turnstile, C3E yagi and 40M rotatable dipole. The yagi and dipole were fixed on 'West' to help reduce interference between other stations at the FD site.

I devised a process to geocode the skimmers and produce a set of files that were plotted using Octave. It's too bad the RBN data doesn't have the skimmer location data included in the archive files. US stations are easy to geocode using callook.info. However, non-US stations require a different approach based on looking up the address in QRZ and geocoding the address using Google. This will require follow on work. For now, I concentrated on the S/N measurements provided by the RBN.

I played with several different graphs and settled on these three because they show three unique contexts. The polar plot provides a sense of strength vs direction. The S/N over time plots show how the bands change throughout the contest. We only ran 2 stations and I think you can see we might have mistimed the switch between 80 and 20. The S/N distribution plots provide a general feel for the strength of the stations's signal over the entire contest period. Note, these plots focus on S/N measurements. It will be interesting to plot all of the contest QSO's to look at overall coverage, but those plots won't have S/N measurements and geocoding may be a problem.

Check out the comparison between W4IY and NR4M. FD2020 NR4M vs W4IY

80 Meter Plots

S/N Polar Plots Distance vs Time Plots S/N Distribution
80M S/N
80M Time
80M S/N Distribution

40 Meter Plots

S/N Polar Plots Distance vs Time Plots S/N Distribution
40M S/N
40M Time
40M S/N Distribution

20 Meter Plots

S/N Polar Plots Distance vs Time Plots S/N Distribution
20M S/N
20M Time
20M S/N Distribution

Methodology

Data from the reverse beacon network were imported into a MySQL database. W4IY spots were singled out by continent (NA), grouped by skimmer and exported to skimmerList.txt. A perl script (geoCode.pl) used callook.info API to geocode the skimmers. The VE station locations were manually built using levinecentral.com lookup. The resulting geocode file (geocodeFile.csv) was imported back into MySql database. A list of all spots for W4IY were created using an INNER JOIN with the skimmer table to produce a list of all W4IY spots with the skimmer geocoded.

Next, the spots were exported to a file (geocodeSpots.cs). A perl script (calcBD.pl) calculated the distance and bearing from the W4IY location to the skimmer. The results were stored in a .csv file (e.g. 80m.csv), which included the epoch time (unix time), s/n distance and bearing to the skimmer.

Using Octave (matlab clone), three graphs for each band were produced. These three graphs were chosen to provide different perspectives of the s/n related data.

Since skimmers are not uniformly distributed throughout the US, the data may be skewed.

Polar Plots

Each S/N measurement was plotted on a set of polar coordinates to show the bearing and S/N level to the skimmer. This shows signal strength per direction. It's interesting to note the station's strong points.

Distance vs Time Plots

This graph shows the distance to the skimmers over the 24 hour contest period. The color of the dots represent the signal strength. Reds and Yellows are stronger S/N measurements than blue. It's interesting to see the band openings, skip zones and operating times.

S/N Distribution

The distribution of S/N measurements are displayed in this graph, which gives you a sense of how strong the stations signal is over the 24 hour period.

Key Code and Notes

Tools: Eclipse IDE with PERL prespective, MySQL Workbench and Octave.

Import Sat/Sunday spot data into MySql using MySQL WorkBench. This takes a long, long time. Over a 1million spots for FD weekend. This is the reason I'm using a database.

In MySQL (using MySQL WorkBench - aka MWB) select all spots for my call, where skimmer is located in NA. Have to do this because geolocation API only looks up US spots. Need to change this so all skimmers are used.

select callsign from spots where dx = "W4IY" and de_cont = "NA" group by callsign;

In MWB, Export the selection to skimmerList.txt

Edit the list to remove the calls with -x suffixes or remove the -x if there is only one callsign wiht the prefix. We're assuming the call signs with the suffix are located at the same place. Note, this removes some spots, but probably not enough to matter.

Execute geoCode.pl, which geocodes the skimmers.

This URL has the US Callbook API:

https://callook.info/api_reference.php

Look up the VE stations manually using this site:

http://www.levinecentral.com/ham/grid_square.php

Note, we really need a way to use this technique to look up all Q's. I'll play around with the ADIF lookup later.

In mysql workbench, clear the table, import geocoded skimmer data into 'skimmers' table. Then, do an inner join.

truncate table skimmers

Import Wizard: file is geocodeFile.csv

select * from skimmers   //verify file came in OK

select callsign, band, db, date, longitude, latitude, gridsquare
    from spots t1 inner join skimmers t2 ON t1.callsign = t2.skimmer
    where dx = 'W4IY';

This provides a table of spots with sn and geocoded skimmers.

Export to geocodeSpots.csv

Edit calcBD.pl to choose the band.

Execute calcBD.pl this will produce a file called 'band.csv'. e.g. 80m.csv with the time (epoch), s/n, dist and bearing to the skimmer. This file can be processed using octave.

Octave

Here is the octave code, which took a long time to figure out!

# Make S/N related graphs
callSign = "NR4M";
band = "80m";

basePath = "/home/stu/Files/eclipse-workspace/beacon/";
cd ([basePath callSign])

fileName = [band ".csv"];
x = csvread(fileName);

# grab the columns to use
t=x(:,1);   # time
s=x(:,2);   # S/N
d=x(:,3);   # distance to skimmer
b=x(:,4);   # bearing to skimmer
theta = b * pi/180;  # convert to radians
tv = datenum(1970,1,1,0,0) + t / 86400; # create time string

# S/N Distribution
f1 = figure(1,"name", [callSign " FD 2020 (" band ")"]);
hist(s,30);
title(["S/N distribution 24 hours (" band ") - " callSign])
grid("on");
axis([0,60]);
x = xlabel("S/N (dB)");
y = ylabel("Number of Measurements");

# Distance vs Time
f2 = figure(2,"name", [callSign " FD 2020 (" band ")"]);
colormap("jet")
scatter(tv,d,10,s,"filled")
caxis([0,60]); # order matters
title(["Distance vs Time with S/N (" band ") - " callSign])
x = xlabel("Time (GMT)");
y = ylabel("Distance (mi)");
datetick("x",15)
axis([737969.75,737970.75]);
#set(x, 'Position', [0.5, -0.05, 0]);
#set(y, 'Position', [-0.0, 0, 0]);
grid("on");

# polar plot
f3 =figure(3,"name", [callSign " FD 2020 Polar (" band ")"])
polar(theta,s, "x")
view (-90,90)       # rotate so 0 is up
axis("ij")          # Flip so west is left
set (gca,"rtick", 10:10:50)  # add more circles
title(["Signal Strength vs Bearing - " callSign])