Note
Click here to download the full example code
Navigating a projectΒΆ
After creating a project and copying in the time data into the time folder, it is useful to be able to navigate a project. This example shows the various types of objects available in resistics that can help navigate a project and access data.
The data in this example has been provided for use by the SAMTEX consortium. For more information, please refer to [Jones2009]. Additional details about the data can be found at https://www.mtnet.info/data/kap03/kap03.html.
from pathlib import Path
import seedir as sd
import plotly
import resistics.letsgo as letsgo
Letβs remind ourselves of the project contents, load the project and have a look at its contents.
project_path = Path("..", "..", "data", "project", "kap03")
sd.seedir(str(project_path), style="emoji")
resenv = letsgo.load(project_path)
Out:
π kap03/
ββπ images/
ββπ time/
β ββπ kap163/
β β ββπ meas01/
β β ββπ data.npy
β β ββπ metadata.json
β ββπ kap160/
β β ββπ meas01/
β β ββπ data.npy
β β ββπ metadata.json
β ββπ kap172/
β ββπ meas01/
β ββπ data.npy
β ββπ metadata.json
ββπ resistics.json
ββπ results/
ββπ calibrate/
ββπ evals/
ββπ masks/
ββπ spectra/
ββπ features/
Project summaries can be quite verbose. Instead, letβs convert it to a pandas DataFrame and see the information in tabular form.
print(resenv.proj.to_dataframe())
Out:
name first_time last_time fs site
0 meas01 2003-10-28 15:30:00 2003-11-24 11:40:30 0.2 kap163
0 meas01 2003-10-28 10:00:00 2003-11-24 15:31:55 0.2 kap160
0 meas01 2003-10-30 13:00:00 2003-11-23 12:41:25 0.2 kap172
The project has three sites, each with a single recording. Another way to look at the sites in the project is to make a list of them.
sites = [site.name for site in resenv.proj]
print(sites)
Out:
['kap163', 'kap160', 'kap172']
To get more information about a single site, get the corresponding Site object.
site = resenv.proj["kap160"]
print(type(site))
Out:
<class 'resistics.project.Site'>
Like most objects in resistics, the Site object has a summary method, which prints out a comprehensive summary of the site.
site.summary()
Out:
{
'dir_path': '../../data/project/kap03/time/kap160',
'measurements': {
'meas01': {
'site_name': 'kap160',
'dir_path': '../../data/project/kap03/time/kap160/meas01',
'metadata': {
'file_info': {
'created_on_local': '2021-07-07T22:26:48.851498',
'created_on_utc': '2021-07-07T21:26:48.851498',
'version': '1.0.0a0'
},
'fs': 0.2,
'chans': ['Hx', 'Hy', 'Hz', 'Ex', 'Ey'],
'n_chans': 5,
'n_samples': 470544,
'first_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'last_time': '2003-11-24 15:31:55.000000_000000_000000_000000',
'system': '',
'serial': '',
'wgs84_latitude': -999.0,
'wgs84_longitude': -999.0,
'easting': -999.0,
'northing': -999.0,
'elevation': -999.0,
'chans_metadata': {
'Hx': {
'name': 'Hx',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hy': {
'name': 'Hy',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hz': {
'name': 'Hz',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ex': {
'name': 'Ex',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ey': {
'name': 'Ey',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
}
},
'history': {'records': []}
},
'reader': {
'name': 'TimeReaderNumpy',
'apply_scalings': True,
'extension': '.npy'
}
}
},
'begin_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'end_time': '2003-11-24 15:31:55.000000_000000_000000_000000'
}
Sometimes, it can be more convenient to access the information from the Site object directly.
print(site.name)
print(site.begin_time)
print(site.end_time)
measurements = [meas.name for meas in site]
print(measurements)
Out:
kap160
2003-10-28 10:00:00
2003-11-24 15:31:55
['meas01']
Itβs also possible to plot the timeline of a single site.
fig = site.plot()
plotly.io.show(fig)
There is only a single measurement in this site named βmeas01β. Letβs get its Measurement object.
meas = site["meas01"]
print(type(meas))
Out:
<class 'resistics.project.Measurement'>
Unsurprisingly, Measurement objects also have a summary method.
meas.summary()
Out:
{
'site_name': 'kap160',
'dir_path': '../../data/project/kap03/time/kap160/meas01',
'metadata': {
'file_info': {
'created_on_local': '2021-07-07T22:26:48.851498',
'created_on_utc': '2021-07-07T21:26:48.851498',
'version': '1.0.0a0'
},
'fs': 0.2,
'chans': ['Hx', 'Hy', 'Hz', 'Ex', 'Ey'],
'n_chans': 5,
'n_samples': 470544,
'first_time': '2003-10-28 10:00:00.000000_000000_000000_000000',
'last_time': '2003-11-24 15:31:55.000000_000000_000000_000000',
'system': '',
'serial': '',
'wgs84_latitude': -999.0,
'wgs84_longitude': -999.0,
'easting': -999.0,
'northing': -999.0,
'elevation': -999.0,
'chans_metadata': {
'Hx': {
'name': 'Hx',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hy': {
'name': 'Hy',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Hz': {
'name': 'Hz',
'data_files': ['data.npy'],
'chan_type': 'magnetic',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ex': {
'name': 'Ex',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
},
'Ey': {
'name': 'Ey',
'data_files': ['data.npy'],
'chan_type': 'electric',
'chan_source': None,
'sensor': '',
'serial': '',
'gain1': 1.0,
'gain2': 1.0,
'scaling': 1.0,
'chopper': False,
'dipole_dist': 1.0,
'sensor_calibration_file': '',
'instrument_calibration_file': ''
}
},
'history': {'records': []}
},
'reader': {
'name': 'TimeReaderNumpy',
'apply_scalings': True,
'extension': '.npy'
}
}
Measurement objects only hold metadata to avoid loading in lots of data when projects are loaded. However, it is possible to read the data from the measurement.
time_data = meas.reader.run(meas.dir_path, metadata=meas.metadata)
time_data.summary()
Out:
##---Begin Summary-------------------------
<class 'resistics.time.TimeData'>
file_info:
created_on_local: '2021-07-07T22:26:48.851498'
created_on_utc: '2021-07-07T21:26:48.851498'
version: 1.0.0a0
fs: 0.2
chans:
- Hx
- Hy
- Hz
- Ex
- Ey
n_chans: 5
n_samples: 470544
first_time: 2003-10-28 10:00:00.000000_000000_000000_000000
last_time: 2003-11-24 15:31:55.000000_000000_000000_000000
system: ''
serial: ''
wgs84_latitude: -999.0
wgs84_longitude: -999.0
easting: -999.0
northing: -999.0
elevation: -999.0
chans_metadata:
Hx:
name: Hx
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Hy:
name: Hy
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Hz:
name: Hz
data_files:
- data.npy
chan_type: magnetic
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Ex:
name: Ex
data_files:
- data.npy
chan_type: electric
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
Ey:
name: Ey
data_files:
- data.npy
chan_type: electric
chan_source: null
sensor: ''
serial: ''
gain1: 1.0
gain2: 1.0
scaling: 1.0
chopper: false
dipole_dist: 1.0
sensor_calibration_file: ''
instrument_calibration_file: ''
history:
records:
- time_local: '2021-10-14T22:19:27.906654'
time_utc: '2021-10-14T22:19:27.906653'
creator:
name: TimeReaderNumpy
apply_scalings: true
extension: .npy
messages:
- Reading raw data from ../../data/project/kap03/time/kap160/meas01
- Sampling frequency 0.2 Hz
- 'From sample, time: 0, 2003-10-28 10:00:00'
- 'To sample, time: 470543, 2003-11-24 15:31:55'
record_type: process
##---End summary---------------------------
Letβs plot the time data.
fig = time_data.plot()
fig.update_layout(height=700)
plotly.io.show(fig)
Total running time of the script: ( 0 minutes 6.065 seconds)