Note
Go to the end to download the full example code
Navigating a project#
After creating a project and copying in the time data into the time folder, it is useful to be able to navigate a project. This example shows the various types of objects available in resistics that can help navigate a project and access data.
The data in this example has been provided for use by the SAMTEX consortium. For more information, please refer to [Jones2009]. Additional details about the data can be found at https://www.mtnet.info/data/kap03/kap03.html.
from pathlib import Path
import seedir as sd
import resistics.letsgo as letsgo
import plotly
Let’s remind ourselves of the project contents, load the project and have a look at its contents.
project_path = Path("..", "..", "data", "project", "kap03")
sd.seedir(str(project_path), style="emoji")
resenv = letsgo.load(project_path)
📁 kap03/
├─📁 spectra/
├─📁 images/
├─📄 resistics.json
├─📁 masks/
├─📁 results/
├─📁 calibrate/
├─📁 evals/
├─📁 features/
└─📁 time/
├─📁 kap172/
│ └─📁 meas01/
│ ├─📄 metadata.json
│ └─📄 data.npy
├─📁 kap163/
│ └─📁 meas01/
│ ├─📄 metadata.json
│ └─📄 data.npy
└─📁 kap160/
└─📁 meas01/
├─📄 metadata.json
└─📄 data.npy
Project summaries can be quite verbose. Instead, let’s convert it to a pandas DataFrame and see the information in tabular form.
print(resenv.proj.to_dataframe())
name fs first_time last_time site
0 meas01 0.2 2003-10-30 13:00:00 2003-11-23 12:41:25 kap172
1 meas01 0.2 2003-10-28 15:30:00 2003-11-24 11:40:30 kap163
2 meas01 0.2 2003-10-28 10:00:00 2003-11-24 15:31:55 kap160
The project has three sites, each with a single recording. Another way to look at the sites in the project is to make a list of them.
['kap172', 'kap163', 'kap160']
To get more information about a single site, get the corresponding Site object.
site = resenv.proj["kap160"]
print(type(site))
<class 'resistics.project.Site'>
Like most objects in resistics, the Site object has a summary method, which prints out a comprehensive summary of the site.
site.summary()
{
'dir_path': '../../data/project/kap03/time/kap160',
'measurements': {
'meas01': {
'site_name': 'kap160',
'dir_path': '../../data/project/kap03/time/kap160/meas01',
'metadata': {
'file_info': {
'created_on_local': '2021-07-07T22:26:48.851498',
'created_on_utc': '2021-07-07T21:26:48.851498',
'version': '1.0.0a0'
},
'fs': 0.2,
'chans': ['Hx', 'Hy', 'Hz', 'Ex', 'Ey'],
'n_chans': 5,
'n_samples': 470544,
'first_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'last_time': '2003-11-24 15:31:55.000000_000000_000000_000000',
'system': '',
'serial': '',
'wgs84_latitude': -999.0,
'wgs84_longitude': -999.0,
'easting': -999.0,
'northing': -999.0,
'elevation': -999.0,
'chans_metadata': {
'Hx': {
'name': 'Hx',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hy': {
'name': 'Hy',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hz': {
'name': 'Hz',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ex': {
'name': 'Ex',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ey': {
'name': 'Ey',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
}
},
'history': {'records': []}
},
'reader': {
'name': 'TimeReaderNumpy',
'apply_scalings': True,
'extension': '.npy'
}
}
},
'begin_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'end_time': '2003-11-24 15:31:55.000000_000000_000000_000000'
}
Sometimes, it can be more convenient to access the information from the Site object directly.
print(site.name)
print(site.begin_time)
print(site.end_time)
measurements = [meas.name for meas in site]
print(measurements)
kap160
2003-10-28 10:00:00
2003-11-24 15:31:55
['meas01']
It’s also possible to plot the timeline of a single site.
fig = site.plot()
plotly.io.show(fig)
There is only a single measurement in this site named “meas01”. Let’s get its Measurement object.
meas = site["meas01"]
print(type(meas))
<class 'resistics.project.Measurement'>
Unsurprisingly, Measurement objects also have a summary method.
meas.summary()
{
'site_name': 'kap160',
'dir_path': '../../data/project/kap03/time/kap160/meas01',
'metadata': {
'file_info': {
'created_on_local': '2021-07-07T22:26:48.851498',
'created_on_utc': '2021-07-07T21:26:48.851498',
'version': '1.0.0a0'
},
'fs': 0.2,
'chans': ['Hx', 'Hy', 'Hz', 'Ex', 'Ey'],
'n_chans': 5,
'n_samples': 470544,
'first_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'last_time': '2003-11-24 15:31:55.000000_000000_000000_000000',
'system': '',
'serial': '',
'wgs84_latitude': -999.0,
'wgs84_longitude': -999.0,
'easting': -999.0,
'northing': -999.0,
'elevation': -999.0,
'chans_metadata': {
'Hx': {
'name': 'Hx',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hy': {
'name': 'Hy',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hz': {
'name': 'Hz',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ex': {
'name': 'Ex',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ey': {
'name': 'Ey',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
}
},
'history': {'records': []}
},
'reader': {
'name': 'TimeReaderNumpy',
'apply_scalings': True,
'extension': '.npy'
}
}
Measurement objects only hold metadata to avoid loading in lots of data when projects are loaded. However, it is possible to read the data from the measurement.
time_data = meas.reader.run(meas.dir_path, metadata=meas.metadata)
time_data.summary()
##---Begin Summary-------------------------
<class 'resistics.time.TimeData'>
file_info:
created_on_local: '2021-07-07T22:26:48.851498'
created_on_utc: '2021-07-07T21:26:48.851498'
version: 1.0.0a0
fs: 0.2
chans:
- Hx
- Hy
- Hz
- Ex
- Ey
n_chans: 5
n_samples: 470544
first_time: 2003-10-28 10:00:00.000000_000000_000000_000000
last_time: 2003-11-24 15:31:55.000000_000000_000000_000000
system: ''
serial: ''
wgs84_latitude: -999.0
wgs84_longitude: -999.0
easting: -999.0
northing: -999.0
elevation: -999.0
chans_metadata:
Hx:
name: Hx
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Hy:
name: Hy
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Hz:
name: Hz
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Ex:
name: Ex
data_files:
- data.npy
chan_type: electric
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Ey:
name: Ey
data_files:
- data.npy
chan_type: electric
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
history:
records:
- time_local: '2023-05-12T14:40:46.976075'
time_utc: '2023-05-12T14:40:46.976074'
creator:
name: TimeReaderNumpy
apply_scalings: true
extension: .npy
messages:
- Reading raw data from ../../data/project/kap03/time/kap160/meas01
- Sampling frequency 0.2 Hz
- 'From sample, time: 0, 2003-10-28 10:00:00'
- 'To sample, time: 470543, 2003-11-24 15:31:55'
record_type: process
##---End summary---------------------------
Let’s plot the time data.
fig = time_data.plot()
fig.update_layout(height=700)
plotly.io.show(fig)
Total running time of the script: ( 0 minutes 7.512 seconds)