resistics.letsgo module¶
This module is the main interface to resistics and includes:
Classes and functions for making, loading and using resistics projects
Functions for processing data
- pydantic model resistics.letsgo.ProjectCreator[source]¶
Bases:
resistics.common.ResisticsProcessProcess to create a project
Show JSON schema
{ "title": "ProjectCreator", "description": "Process to create a project", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "dir_path": { "title": "Dir Path", "type": "string", "format": "path" }, "metadata": { "$ref": "#/definitions/ProjectMetadata" } }, "required": [ "dir_path", "metadata" ], "definitions": { "ResisticsFile": { "title": "ResisticsFile", "description": "Required information for writing out a resistics file", "type": "object", "properties": { "created_on_local": { "title": "Created On Local", "type": "string", "format": "date-time" }, "created_on_utc": { "title": "Created On Utc", "type": "string", "format": "date-time" }, "version": { "title": "Version", "type": "string" } } }, "ProjectMetadata": { "title": "ProjectMetadata", "description": "Project metadata", "type": "object", "properties": { "file_info": { "$ref": "#/definitions/ResisticsFile" }, "ref_time": { "title": "Ref Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "location": { "title": "Location", "default": "", "type": "string" }, "country": { "title": "Country", "default": "", "type": "string" }, "year": { "title": "Year", "default": -999, "type": "integer" }, "description": { "title": "Description", "default": "", "type": "string" }, "contributors": { "title": "Contributors", "default": [], "type": "array", "items": { "type": "string" } } }, "required": [ "ref_time" ] } } }
- field dir_path: pathlib.Path [Required]¶
- field metadata: resistics.project.ProjectMetadata [Required]¶
- run()[source]¶
Create the project
- Raises
ProjectCreateError – If an existing project found
- resistics.letsgo.new(dir_path: Union[pathlib.Path, str], proj_info: Dict[str, Any]) bool[source]¶
Create a new project
- Parameters
dir_path (Union[Path, str]) – The directory to create the project in
proj_info (Dict[str, Any]) – Any project details
- Returns
True if the creator was successful
- Return type
bool
- pydantic model resistics.letsgo.ProjectLoader[source]¶
Bases:
resistics.common.ResisticsProcessProject loader
Show JSON schema
{ "title": "ProjectLoader", "description": "Project loader", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "dir_path": { "title": "Dir Path", "type": "string", "format": "path" } }, "required": [ "dir_path" ] }
- field dir_path: pathlib.Path [Required]¶
- run(config: resistics.config.Configuration) resistics.project.Project[source]¶
Load a project
- Parameters
config (Configuration) – The configuration for the purposes of getting the time readers
- Returns
Project instance
- Return type
- Raises
ProjectLoadError – If the resistcs project metadata is not found
- pydantic model resistics.letsgo.ResisticsEnvironment[source]¶
Bases:
resistics.common.ResisticsModelA Resistics environment which combines a project and a configuration
Show JSON schema
{ "title": "ResisticsEnvironment", "description": "A Resistics environment which combines a project and a configuration", "type": "object", "properties": { "proj": { "$ref": "#/definitions/Project" }, "config": { "$ref": "#/definitions/Configuration" } }, "required": [ "proj", "config" ], "definitions": { "ResisticsFile": { "title": "ResisticsFile", "description": "Required information for writing out a resistics file", "type": "object", "properties": { "created_on_local": { "title": "Created On Local", "type": "string", "format": "date-time" }, "created_on_utc": { "title": "Created On Utc", "type": "string", "format": "date-time" }, "version": { "title": "Version", "type": "string" } } }, "ProjectMetadata": { "title": "ProjectMetadata", "description": "Project metadata", "type": "object", "properties": { "file_info": { "$ref": "#/definitions/ResisticsFile" }, "ref_time": { "title": "Ref Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "location": { "title": "Location", "default": "", "type": "string" }, "country": { "title": "Country", "default": "", "type": "string" }, "year": { "title": "Year", "default": -999, "type": "integer" }, "description": { "title": "Description", "default": "", "type": "string" }, "contributors": { "title": "Contributors", "default": [], "type": "array", "items": { "type": "string" } } }, "required": [ "ref_time" ] }, "ChanMetadata": { "title": "ChanMetadata", "description": "Channel metadata", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "data_files": { "title": "Data Files", "type": "array", "items": { "type": "string" } }, "chan_type": { "title": "Chan Type", "type": "string" }, "chan_source": { "title": "Chan Source", "type": "string" }, "sensor": { "title": "Sensor", "default": "", "type": "string" }, "serial": { "title": "Serial", "default": "", "type": "string" }, "gain1": { "title": "Gain1", "default": 1, "type": "number" }, "gain2": { "title": "Gain2", "default": 1, "type": "number" }, "scaling": { "title": "Scaling", "default": 1, "type": "number" }, "chopper": { "title": "Chopper", "default": false, "type": "boolean" }, "dipole_dist": { "title": "Dipole Dist", "default": 1, "type": "number" }, "sensor_calibration_file": { "title": "Sensor Calibration File", "default": "", "type": "string" }, "instrument_calibration_file": { "title": "Instrument Calibration File", "default": "", "type": "string" } }, "required": [ "name" ] }, "Record": { "title": "Record", "description": "Class to hold a record\n\nA record holds information about a process that was run. It is intended to\ntrack processes applied to data, allowing a process history to be saved\nalong with any datasets.\n\nExamples\n--------\nA simple example of creating a process record\n\n>>> from resistics.common import Record\n>>> messages = [\"message 1\", \"message 2\"]\n>>> record = Record(\n... creator={\"name\": \"example\", \"parameter1\": 15},\n... messages=messages,\n... record_type=\"example\"\n... )\n>>> record.summary()\n{\n 'time_local': '...',\n 'time_utc': '...',\n 'creator': {'name': 'example', 'parameter1': 15},\n 'messages': ['message 1', 'message 2'],\n 'record_type': 'example'\n}", "type": "object", "properties": { "time_local": { "title": "Time Local", "type": "string", "format": "date-time" }, "time_utc": { "title": "Time Utc", "type": "string", "format": "date-time" }, "creator": { "title": "Creator", "type": "object" }, "messages": { "title": "Messages", "type": "array", "items": { "type": "string" } }, "record_type": { "title": "Record Type", "type": "string" } }, "required": [ "creator", "messages", "record_type" ] }, "History": { "title": "History", "description": "Class for storing processing history\n\nParameters\n----------\nrecords : List[Record], optional\n List of records, by default []\n\nExamples\n--------\n>>> from resistics.testing import record_example1, record_example2\n>>> from resistics.common import History\n>>> record1 = record_example1()\n>>> record2 = record_example2()\n>>> history = History(records=[record1, record2])\n>>> history.summary()\n{\n 'records': [\n {\n 'time_local': '...',\n 'time_utc': '...',\n 'creator': {\n 'name': 'example1',\n 'a': 5,\n 'b': -7.0\n },\n 'messages': ['Message 1', 'Message 2'],\n 'record_type': 'process'\n },\n {\n 'time_local': '...',\n 'time_utc': '...',\n 'creator': {\n 'name': 'example2',\n 'a': 'parzen',\n 'b': -21\n },\n 'messages': ['Message 5', 'Message 6'],\n 'record_type': 'process'\n }\n ]\n}", "type": "object", "properties": { "records": { "title": "Records", "default": [], "type": "array", "items": { "$ref": "#/definitions/Record" } } } }, "TimeMetadata": { "title": "TimeMetadata", "description": "Time metadata", "type": "object", "properties": { "file_info": { "$ref": "#/definitions/ResisticsFile" }, "fs": { "title": "Fs", "type": "number" }, "chans": { "title": "Chans", "type": "array", "items": { "type": "string" } }, "n_chans": { "title": "N Chans", "type": "integer" }, "n_samples": { "title": "N Samples", "type": "integer" }, "first_time": { "title": "First Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "last_time": { "title": "Last Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "system": { "title": "System", "default": "", "type": "string" }, "serial": { "title": "Serial", "default": "", "type": "string" }, "wgs84_latitude": { "title": "Wgs84 Latitude", "default": -999.0, "type": "number" }, "wgs84_longitude": { "title": "Wgs84 Longitude", "default": -999.0, "type": "number" }, "easting": { "title": "Easting", "default": -999.0, "type": "number" }, "northing": { "title": "Northing", "default": -999.0, "type": "number" }, "elevation": { "title": "Elevation", "default": -999.0, "type": "number" }, "chans_metadata": { "title": "Chans Metadata", "type": "object", "additionalProperties": { "$ref": "#/definitions/ChanMetadata" } }, "history": { "title": "History", "default": { "records": [] }, "allOf": [ { "$ref": "#/definitions/History" } ] } }, "required": [ "fs", "chans", "n_samples", "first_time", "last_time", "chans_metadata" ] }, "TimeReader": { "title": "TimeReader", "description": "Base class for resistics processes\n\nResistics processes perform operations on data (including read and write\noperations). Each time a ResisticsProcess child class is run, it should add\na process record to the dataset", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "apply_scalings": { "title": "Apply Scalings", "default": true, "type": "boolean" }, "extension": { "title": "Extension", "type": "string" } } }, "Measurement": { "title": "Measurement", "description": "Class for interfacing with a measurement\n\nThe class holds the original time series metadata and can provide\ninformation about other types of data", "type": "object", "properties": { "site_name": { "title": "Site Name", "type": "string" }, "dir_path": { "title": "Dir Path", "type": "string", "format": "path" }, "metadata": { "$ref": "#/definitions/TimeMetadata" }, "reader": { "$ref": "#/definitions/TimeReader" } }, "required": [ "site_name", "dir_path", "metadata", "reader" ] }, "Site": { "title": "Site", "description": "Class for describing Sites\n\n.. note::\n\n This should essentially describe a single instrument setup. If the same\n site is re-occupied later with a different instrument setup, it is\n suggested to split this into a different site.", "type": "object", "properties": { "dir_path": { "title": "Dir Path", "type": "string", "format": "path" }, "measurements": { "title": "Measurements", "type": "object", "additionalProperties": { "$ref": "#/definitions/Measurement" } }, "begin_time": { "title": "Begin Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "end_time": { "title": "End Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] } }, "required": [ "dir_path", "measurements", "begin_time", "end_time" ] }, "Project": { "title": "Project", "description": "Class to describe a resistics project\n\nThe resistics Project Class connects all resistics data. It is an essential\npart of processing data with resistics.\n\nResistics projects are in directory with several sub-directories. Project\nmetadata is saved in the resistics.json file at the top level directory.", "type": "object", "properties": { "dir_path": { "title": "Dir Path", "type": "string", "format": "path" }, "begin_time": { "title": "Begin Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "end_time": { "title": "End Time", "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v", "examples": [ "2021-01-01 00:00:00.000061_035156_250000_000000" ] }, "metadata": { "$ref": "#/definitions/ProjectMetadata" }, "sites": { "title": "Sites", "default": {}, "type": "object", "additionalProperties": { "$ref": "#/definitions/Site" } } }, "required": [ "dir_path", "begin_time", "end_time", "metadata" ] }, "TimeProcess": { "title": "TimeProcess", "description": "Parent class for processing time data", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "DecimationSetup": { "title": "DecimationSetup", "description": "Process to calculate decimation parameters\n\nParameters\n----------\nn_levels : int, optional\n Number of decimation levels, by default 8\nper_level : int, optional\n Number of frequencies per level, by default 5\nmin_samples : int, optional\n Number of samples under which to quit decimating\ndiv_factor : int, optional\n Minimum division factor for decimation, by default 2.\neval_freqs : Optional[List[float]], optional\n Explicit definition of evaluation frequencies as a flat list, by\n default None. Must be of size n_levels * per_level\n\nExamples\n--------\n>>> from resistics.decimate import DecimationSetup\n>>> dec_setup = DecimationSetup(n_levels=3, per_level=2)\n>>> dec_params = dec_setup.run(128)\n>>> print(dec_params.to_dataframe())\n 0 1 fs factors increments\ndecimation level\n0 32.0 22.627417 128.0 1 1\n1 16.0 11.313708 64.0 2 2\n2 8.0 5.656854 32.0 4 2", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "n_levels": { "title": "N Levels", "default": 8, "type": "integer" }, "per_level": { "title": "Per Level", "default": 5, "type": "integer" }, "min_samples": { "title": "Min Samples", "default": 256, "type": "integer" }, "div_factor": { "title": "Div Factor", "default": 2, "type": "integer" }, "eval_freqs": { "title": "Eval Freqs", "type": "array", "items": { "type": "number" } } } }, "Decimator": { "title": "Decimator", "description": "Decimate the time data into multiple levels\n\nThere are two options for decimation, using time data Resample or using\ntime data Decimate. The default is to use Resample.", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "resample": { "title": "Resample", "default": true, "type": "boolean" }, "max_single_factor": { "title": "Max Single Factor", "default": 3, "minimum": 3, "type": "integer" } } }, "WindowSetup": { "title": "WindowSetup", "description": "Setup WindowParameters\n\nWindowSetup outputs the WindowParameters to use for windowing decimated\ntime data.\n\nWindow parameters are simply the window and overlap sizes for each\ndecimation level.\n\nParameters\n----------\nmin_size : int, optional\n Minimum window size, by default 128\nmin_olap : int, optional\n Minimum overlap size, by default 32\nwin_factor : int, optional\n Window factor, by default 4. Window sizes are calculated by sampling\n frequency / 4 to ensure sufficient frequency resolution. If the\n sampling frequency is small, window size will be adjusted to\n min_size\nolap_proportion : float, optional\n The proportion of the window size to use as the overlap, by default\n 0.25. For example, for a window size of 128, the overlap would be\n 0.25 * 128 = 32\nmin_n_wins : int, optional\n The minimum number of windows needed in a decimation level, by\n default 5\nwin_sizes : Optional[List[int]], optional\n Explicit define window sizes, by default None. Must have the same\n length as number of decimation levels\nolap_sizes : Optional[List[int]], optional\n Explicitly define overlap sizes, by default None. Must have the same\n length as number of decimation levels\n\nExamples\n--------\nGenerate decimation and windowing parameters for data sampled at 0.05 Hz or\n20 seconds sampling period\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.window import WindowSetup\n>>> dec_params = DecimationSetup(n_levels=3, per_level=3).run(0.05)\n>>> dec_params.summary()\n{\n 'fs': 0.05,\n 'n_levels': 3,\n 'per_level': 3,\n 'min_samples': 256,\n 'eval_freqs': [\n 0.0125,\n 0.008838834764831844,\n 0.00625,\n 0.004419417382415922,\n 0.003125,\n 0.002209708691207961,\n 0.0015625,\n 0.0011048543456039805,\n 0.00078125\n ],\n 'dec_factors': [1, 2, 8],\n 'dec_increments': [1, 2, 4],\n 'dec_fs': [0.05, 0.025, 0.00625]\n}\n>>> win_params = WindowSetup().run(dec_params.n_levels, dec_params.dec_fs)\n>>> win_params.summary()\n{\n 'n_levels': 3,\n 'min_n_wins': 5,\n 'win_sizes': [128, 128, 128],\n 'olap_sizes': [32, 32, 32]\n}\n\nWindow parameters can also be explicitly defined\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.window import WindowSetup\n>>> dec_setup = DecimationSetup(n_levels=3, per_level=3)\n>>> dec_params = dec_setup.run(0.05)\n>>> win_setup = WindowSetup(win_sizes=[1000, 578, 104])\n>>> win_params = win_setup.run(dec_params.n_levels, dec_params.dec_fs)\n>>> win_params.summary()\n{\n 'n_levels': 3,\n 'min_n_wins': 5,\n 'win_sizes': [1000, 578, 104],\n 'olap_sizes': [250, 144, 32]\n}", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "min_size": { "title": "Min Size", "default": 128, "type": "integer" }, "min_olap": { "title": "Min Olap", "default": 32, "type": "integer" }, "win_factor": { "title": "Win Factor", "default": 4, "type": "integer" }, "olap_proportion": { "title": "Olap Proportion", "default": 0.25, "type": "number" }, "min_n_wins": { "title": "Min N Wins", "default": 5, "type": "integer" }, "win_sizes": { "title": "Win Sizes", "type": "array", "items": { "type": "integer" } }, "olap_sizes": { "title": "Olap Sizes", "type": "array", "items": { "type": "integer" } } } }, "Windower": { "title": "Windower", "description": "Windows DecimatedData\n\nThis is the primary window making process for resistics and should be used\nwhen alignment of windows with a site or across sites is required.\n\nThis method uses numpy striding to produce window views into the decimated\ndata.\n\nSee Also\n--------\nWindowerTarget : A windower to make a target number of windows\n\nExamples\n--------\nThe Windower windows a DecimatedData object given a reference time and some\nwindow parameters.\n\nThere's quite a few imports needed for this example. Begin by doing the\nimports, defining a reference time and generating random decimated data.\n\n>>> from resistics.sampling import to_datetime\n>>> from resistics.testing import decimated_data_linear\n>>> from resistics.window import WindowSetup, Windower\n>>> dec_data = decimated_data_linear(fs=128)\n>>> ref_time = dec_data.metadata.first_time\n>>> print(dec_data.to_string())\n<class 'resistics.decimate.DecimatedData'>\n fs dt n_samples first_time last_time\nlevel\n0 2048.0 0.000488 16384 2021-01-01 00:00:00 2021-01-01 00:00:07.99951171875\n1 512.0 0.001953 4096 2021-01-01 00:00:00 2021-01-01 00:00:07.998046875\n2 128.0 0.007812 1024 2021-01-01 00:00:00 2021-01-01 00:00:07.9921875\n\nNext, initialise the window parameters. For this example, use small windows,\nwhich will make inspecting them easier.\n\n>>> win_params = WindowSetup(win_sizes=[16,16,16], min_olap=4).run(dec_data.metadata.n_levels, dec_data.metadata.fs)\n>>> win_params.summary()\n{\n 'n_levels': 3,\n 'min_n_wins': 5,\n 'win_sizes': [16, 16, 16],\n 'olap_sizes': [4, 4, 4]\n}\n\nPerform the windowing. This actually creates views into the decimated data\nusing the numpy.lib.stride_tricks.sliding_window_view function. The shape\nfor a data array at a decimation level is: n_wins x n_chans x win_size. The\ninformation about each level is also in the levels_metadata attribute of\nWindowedMetadata.\n\n>>> win_data = Windower().run(ref_time, win_params, dec_data)\n>>> win_data.data[0].shape\n(1365, 2, 16)\n>>> for level_metadata in win_data.metadata.levels_metadata:\n... level_metadata.summary()\n{\n 'fs': 2048.0,\n 'n_wins': 1365,\n 'win_size': 16,\n 'olap_size': 4,\n 'index_offset': 0\n}\n{\n 'fs': 512.0,\n 'n_wins': 341,\n 'win_size': 16,\n 'olap_size': 4,\n 'index_offset': 0\n}\n{\n 'fs': 128.0,\n 'n_wins': 85,\n 'win_size': 16,\n 'olap_size': 4,\n 'index_offset': 0\n}\n\nLet's look at an example of data from the first decimation level for the\nfirst channel. This is simply a linear set of data ranging from 0...16_383.\n\n>>> dec_data.data[0][0]\narray([ 0, 1, 2, ..., 16381, 16382, 16383])\n\nInspecting the first few windows shows they are as expected including the\noverlap.\n\n>>> win_data.data[0][0, 0]\narray([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])\n>>> win_data.data[0][1, 0]\narray([12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27])\n>>> win_data.data[0][2, 0]\narray([24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39])", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "FourierTransform": { "title": "FourierTransform", "description": "Perform a Fourier transform of the windowed data\n\nThe processor is inspired by the scipy.signal.stft function which performs\na similar process and involves a Fourier transform along the last axis of\nthe windowed data.\n\nParameters\n----------\nwin_fnc : Union[str, Tuple[str, float]]\n The window to use before performing the FFT, by default (\"kaiser\", 14)\ndetrend : Union[str, None]\n Type of detrending to apply before performing FFT, by default linear\n detrend. Setting to None will not apply any detrending to the data prior\n to the FFT\nworkers : int\n The number of CPUs to use, by default max - 2\n\nExamples\n--------\nThis example will get periodic decimated data, perfrom windowing and run the\nFourier transform on the windowed data.\n\n.. plot::\n :width: 90%\n\n >>> import matplotlib.pyplot as plt\n >>> import numpy as np\n >>> from resistics.testing import decimated_data_periodic\n >>> from resistics.window import WindowSetup, Windower\n >>> from resistics.spectra import FourierTransform\n >>> frequencies = {\"chan1\": [870, 590, 110, 32, 12], \"chan2\": [480, 375, 210, 60, 45]}\n >>> dec_data = decimated_data_periodic(frequencies, fs=128)\n >>> dec_data.metadata.chans\n ['chan1', 'chan2']\n >>> print(dec_data.to_string())\n <class 'resistics.decimate.DecimatedData'>\n fs dt n_samples first_time last_time\n level\n 0 2048.0 0.000488 16384 2021-01-01 00:00:00 2021-01-01 00:00:07.99951171875\n 1 512.0 0.001953 4096 2021-01-01 00:00:00 2021-01-01 00:00:07.998046875\n 2 128.0 0.007812 1024 2021-01-01 00:00:00 2021-01-01 00:00:07.9921875\n\n Perform the windowing\n\n >>> win_params = WindowSetup().run(dec_data.metadata.n_levels, dec_data.metadata.fs)\n >>> win_data = Windower().run(dec_data.metadata.first_time, win_params, dec_data)\n\n And then the Fourier transform. By default, the data will be (linearly)\n detrended and mutliplied by a Kaiser window prior to the Fourier\n transform\n\n >>> spec_data = FourierTransform().run(win_data)\n\n For plotting of magnitude, let's stack the spectra\n\n >>> freqs_0 = spec_data.metadata.levels_metadata[0].freqs\n >>> data_0 = np.absolute(spec_data.data[0]).mean(axis=0)\n >>> freqs_1 = spec_data.metadata.levels_metadata[1].freqs\n >>> data_1 = np.absolute(spec_data.data[1]).mean(axis=0)\n >>> freqs_2 = spec_data.metadata.levels_metadata[2].freqs\n >>> data_2 = np.absolute(spec_data.data[2]).mean(axis=0)\n\n Now plot\n\n >>> plt.subplot(3,1,1) # doctest: +SKIP\n >>> plt.plot(freqs_0, data_0[0], label=\"chan1\") # doctest: +SKIP\n >>> plt.plot(freqs_0, data_0[1], label=\"chan2\") # doctest: +SKIP\n >>> plt.grid()\n >>> plt.title(\"Decimation level 0\") # doctest: +SKIP\n >>> plt.legend() # doctest: +SKIP\n >>> plt.subplot(3,1,2) # doctest: +SKIP\n >>> plt.plot(freqs_1, data_1[0], label=\"chan1\") # doctest: +SKIP\n >>> plt.plot(freqs_1, data_1[1], label=\"chan2\") # doctest: +SKIP\n >>> plt.grid()\n >>> plt.title(\"Decimation level 1\") # doctest: +SKIP\n >>> plt.legend() # doctest: +SKIP\n >>> plt.subplot(3,1,3) # doctest: +SKIP\n >>> plt.plot(freqs_2, data_2[0], label=\"chan1\") # doctest: +SKIP\n >>> plt.plot(freqs_2, data_2[1], label=\"chan2\") # doctest: +SKIP\n >>> plt.grid()\n >>> plt.title(\"Decimation level 2\") # doctest: +SKIP\n >>> plt.legend() # doctest: +SKIP\n >>> plt.xlabel(\"Frequency\") # doctest: +SKIP\n >>> plt.tight_layout() # doctest: +SKIP\n >>> plt.show() # doctest: +SKIP", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "win_fnc": { "title": "Win Fnc", "default": [ "kaiser", 14 ], "anyOf": [ { "type": "string" }, { "type": "array", "items": [ { "type": "string" }, { "type": "number" } ] } ] }, "detrend": { "title": "Detrend", "default": "linear", "type": "string" }, "workers": { "title": "Workers", "default": -2, "type": "integer" } } }, "SpectraProcess": { "title": "SpectraProcess", "description": "Parent class for spectra processes", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "EvaluationFreqs": { "title": "EvaluationFreqs", "description": "Calculate the spectra values at the evaluation frequencies\n\nThis is done using linear interpolation in the complex domain\n\nExample\n-------\nThe example will show interpolation to evaluation frequencies on a very\nsimple example. Begin by generating some example spectra data.\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.spectra import EvaluationFreqs\n>>> from resistics.testing import spectra_data_basic\n>>> spec_data = spectra_data_basic()\n>>> spec_data.metadata.n_levels\n1\n>>> spec_data.metadata.chans\n['chan1']\n>>> spec_data.metadata.levels_metadata[0].summary()\n{\n 'fs': 180.0,\n 'n_wins': 2,\n 'win_size': 20,\n 'olap_size': 5,\n 'index_offset': 0,\n 'n_freqs': 10,\n 'freqs': [0.0, 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0]\n}\n\nThe spectra data has only a single channel and a single level which has 2\nwindows. Now define our evaluation frequencies.\n\n>>> eval_freqs = [1, 12, 23, 34, 45, 56, 67, 78, 89]\n>>> dec_setup = DecimationSetup(n_levels=1, per_level=9, eval_freqs=eval_freqs)\n>>> dec_params = dec_setup.run(spec_data.metadata.fs[0])\n>>> dec_params.summary()\n{\n 'fs': 180.0,\n 'n_levels': 1,\n 'per_level': 9,\n 'min_samples': 256,\n 'eval_freqs': [1.0, 12.0, 23.0, 34.0, 45.0, 56.0, 67.0, 78.0, 89.0],\n 'dec_factors': [1],\n 'dec_increments': [1],\n 'dec_fs': [180.0]\n}\n\nNow calculate the spectra at the evaluation frequencies\n\n>>> eval_data = EvaluationFreqs().run(dec_params, spec_data)\n>>> eval_data.metadata.levels_metadata[0].summary()\n{\n 'fs': 180.0,\n 'n_wins': 2,\n 'win_size': 20,\n 'olap_size': 5,\n 'index_offset': 0,\n 'n_freqs': 9,\n 'freqs': [1.0, 12.0, 23.0, 34.0, 45.0, 56.0, 67.0, 78.0, 89.0]\n}\n\nTo double check everything is as expected, let's compare the data. Comparing\nwindow 1 gives\n\n>>> print(spec_data.data[0][0, 0])\n[0.+0.j 1.+1.j 2.+2.j 3.+3.j 4.+4.j 5.+5.j 6.+6.j 7.+7.j 8.+8.j 9.+9.j]\n>>> print(eval_data.data[0][0, 0])\n[0.1+0.1j 1.2+1.2j 2.3+2.3j 3.4+3.4j 4.5+4.5j 5.6+5.6j 6.7+6.7j 7.8+7.8j\n 8.9+8.9j]\n\nAnd window 2\n\n>>> print(spec_data.data[0][1, 0])\n[-1. +1.j 0. +2.j 1. +3.j 2. +4.j 3. +5.j 4. +6.j 5. +7.j 6. +8.j\n 7. +9.j 8.+10.j]\n>>> print(eval_data.data[0][1, 0])\n[-0.9+1.1j 0.2+2.2j 1.3+3.3j 2.4+4.4j 3.5+5.5j 4.6+6.6j 5.7+7.7j\n 6.8+8.8j 7.9+9.9j]", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "Calibrator": { "title": "Calibrator", "description": "Parent class for a calibrator", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "chans": { "title": "Chans", "type": "array", "items": { "type": "string" } } } }, "TransferFunction": { "title": "TransferFunction", "description": "Define a generic transfer function\n\nThis class is a describes generic transfer function, including:\n\n- The output channels for the transfer function\n- The input channels for the transfer function\n- The cross channels for the transfer function\n\nThe cross channels are the channels that will be used to calculate out the\ncross powers for the regression.\n\nThis generic parent class has no implemented plotting function. However,\nchild classes may have a plotting function as different transfer functions\nmay need different types of plots.\n\n.. note::\n\n Users interested in writing a custom transfer function should inherit\n from this generic Transfer function\n\nSee Also\n--------\nImpandanceTensor : Transfer function for the MT impedance tensor\nTipper : Transfer function for the MT tipper\n\nExamples\n--------\nA generic example\n\n>>> tf = TransferFunction(variation=\"example\", out_chans=[\"bye\", \"see you\", \"ciao\"], in_chans=[\"hello\", \"hi_there\"])\n>>> print(tf.to_string())\n| bye | | bye_hello bye_hi_there | | hello |\n| see you | = | see you_hello see you_hi_there | | hi_there |\n| ciao | | ciao_hello ciao_hi_there |\n\nCombining the impedance tensor and the tipper into one TransferFunction\n\n>>> tf = TransferFunction(variation=\"combined\", out_chans=[\"Ex\", \"Ey\"], in_chans=[\"Hx\", \"Hy\", \"Hz\"])\n>>> print(tf.to_string())\n| Ex | | Ex_Hx Ex_Hy Ex_Hz | | Hx |\n| Ey | = | Ey_Hx Ey_Hy Ey_Hz | | Hy |\n | Hz |", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "variation": { "title": "Variation", "default": "generic", "maxLength": 16, "type": "string" }, "out_chans": { "title": "Out Chans", "type": "array", "items": { "type": "string" } }, "in_chans": { "title": "In Chans", "type": "array", "items": { "type": "string" } }, "cross_chans": { "title": "Cross Chans", "type": "array", "items": { "type": "string" } }, "n_out": { "title": "N Out", "type": "integer" }, "n_in": { "title": "N In", "type": "integer" }, "n_cross": { "title": "N Cross", "type": "integer" } }, "required": [ "out_chans", "in_chans" ] }, "RegressionPreparerGathered": { "title": "RegressionPreparerGathered", "description": "Regression preparer for gathered data\n\nIn nearly all cases, this is the regresson preparer to use. As input, it\nrequires GatheredData.", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "Solver": { "title": "Solver", "description": "General resistics solver", "type": "object", "properties": { "name": { "title": "Name", "type": "string" } } }, "Configuration": { "title": "Configuration", "description": "The resistics configuration\n\nConfiguration can be customised by users who wish to use their own custom\nprocesses for certain steps. In most cases, customisation will be for:\n\n- Implementing new time data readers\n- Implementing readers for specific calibration formats\n- Implementing time data processors\n- Implementing spectra data processors\n- Adding new features to extract from the data\n\nExamples\n--------\nFrequently, configuration will be used to change data readers.\n\n>>> from resistics.letsgo import get_default_configuration\n>>> config = get_default_configuration()\n>>> config.name\n'default'\n>>> for tr in config.time_readers:\n... tr.summary()\n{\n 'name': 'TimeReaderAscii',\n 'apply_scalings': True,\n 'extension': '.txt',\n 'delimiter': None,\n 'n_header': 0\n}\n{\n 'name': 'TimeReaderNumpy',\n 'apply_scalings': True,\n 'extension': '.npy'\n}\n>>> config.sensor_calibrator.summary()\n{\n 'name': 'SensorCalibrator',\n 'chans': None,\n 'readers': [\n {\n 'name': 'SensorCalibrationJSON',\n 'extension': '.json',\n 'file_str': 'IC_$sensor$extension'\n }\n ]\n}\n\nTo change these, it's best to make a new configuration with a different name\n\n>>> from resistics.letsgo import Configuration\n>>> from resistics.time import TimeReaderNumpy\n>>> config = Configuration(name=\"myconfig\", time_readers=[TimeReaderNumpy(apply_scalings=False)])\n>>> for tr in config.time_readers:\n... tr.summary()\n{\n 'name': 'TimeReaderNumpy',\n 'apply_scalings': False,\n 'extension': '.npy'\n}\n\nOr for the sensor calibration\n\n>>> from resistics.calibrate import SensorCalibrator, SensorCalibrationTXT\n>>> calibration_reader = SensorCalibrationTXT(file_str=\"lemi120_IC_$serial$extension\")\n>>> calibrator = SensorCalibrator(chans=[\"Hx\", \"Hy\", \"Hz\"], readers=[calibration_reader])\n>>> config = Configuration(name=\"myconfig\", sensor_calibrator=calibrator)\n>>> config.sensor_calibrator.summary()\n{\n 'name': 'SensorCalibrator',\n 'chans': ['Hx', 'Hy', 'Hz'],\n 'readers': [\n {\n 'name': 'SensorCalibrationTXT',\n 'extension': '.TXT',\n 'file_str': 'lemi120_IC_$serial$extension'\n }\n ]\n}\n\nAs a final example, create a configuration which used targetted windowing\ninstead of specified window sizes\n\n>>> from resistics.letsgo import Configuration\n>>> from resistics.window import WindowerTarget\n>>> config = Configuration(name=\"window_target\", windower=WindowerTarget(target=500))\n>>> config.name\n'window_target'\n>>> config.windower.summary()\n{\n 'name': 'WindowerTarget',\n 'target': 500,\n 'min_size': 64,\n 'olap_proportion': 0.25\n}", "type": "object", "properties": { "name": { "title": "Name", "type": "string" }, "time_readers": { "title": "Time Readers", "default": [ { "name": "TimeReaderAscii", "apply_scalings": true, "extension": ".txt", "delimiter": null, "n_header": 0 }, { "name": "TimeReaderNumpy", "apply_scalings": true, "extension": ".npy" } ], "type": "array", "items": { "$ref": "#/definitions/TimeReader" } }, "time_processors": { "title": "Time Processors", "default": [ { "name": "InterpolateNans" }, { "name": "RemoveMean" } ], "type": "array", "items": { "$ref": "#/definitions/TimeProcess" } }, "dec_setup": { "title": "Dec Setup", "default": { "name": "DecimationSetup", "n_levels": 8, "per_level": 5, "min_samples": 256, "div_factor": 2, "eval_freqs": null }, "allOf": [ { "$ref": "#/definitions/DecimationSetup" } ] }, "decimator": { "title": "Decimator", "default": { "name": "Decimator", "resample": true, "max_single_factor": 3 }, "allOf": [ { "$ref": "#/definitions/Decimator" } ] }, "win_setup": { "title": "Win Setup", "default": { "name": "WindowSetup", "min_size": 128, "min_olap": 32, "win_factor": 4, "olap_proportion": 0.25, "min_n_wins": 5, "win_sizes": null, "olap_sizes": null }, "allOf": [ { "$ref": "#/definitions/WindowSetup" } ] }, "windower": { "title": "Windower", "default": { "name": "Windower" }, "allOf": [ { "$ref": "#/definitions/Windower" } ] }, "fourier": { "title": "Fourier", "default": { "name": "FourierTransform", "win_fnc": [ "kaiser", 14 ], "detrend": "linear", "workers": -2 }, "allOf": [ { "$ref": "#/definitions/FourierTransform" } ] }, "spectra_processors": { "title": "Spectra Processors", "default": [], "type": "array", "items": { "$ref": "#/definitions/SpectraProcess" } }, "evals": { "title": "Evals", "default": { "name": "EvaluationFreqs" }, "allOf": [ { "$ref": "#/definitions/EvaluationFreqs" } ] }, "sensor_calibrator": { "title": "Sensor Calibrator", "default": { "name": "SensorCalibrator", "chans": null, "readers": [ { "name": "SensorCalibrationJSON", "extension": ".json", "file_str": "IC_$sensor$extension" } ] }, "allOf": [ { "$ref": "#/definitions/Calibrator" } ] }, "tf": { "title": "Tf", "default": { "name": "ImpedanceTensor", "variation": "default", "out_chans": [ "Ex", "Ey" ], "in_chans": [ "Hx", "Hy" ], "cross_chans": [ "Hx", "Hy" ], "n_out": 2, "n_in": 2, "n_cross": 2 }, "allOf": [ { "$ref": "#/definitions/TransferFunction" } ] }, "regression_preparer": { "title": "Regression Preparer", "default": { "name": "RegressionPreparerGathered" }, "allOf": [ { "$ref": "#/definitions/RegressionPreparerGathered" } ] }, "solver": { "title": "Solver", "default": { "name": "SolverScikitTheilSen", "fit_intercept": false, "normalize": false, "n_jobs": -2, "max_subpopulation": 2000, "n_subsamples": null }, "allOf": [ { "$ref": "#/definitions/Solver" } ] } }, "required": [ "name" ] } } }
- field proj: resistics.project.Project [Required]¶
The project
- field config: resistics.config.Configuration [Required]¶
The configuration for processing
- resistics.letsgo.load(dir_path: Union[pathlib.Path, str], config: Optional[resistics.config.Configuration] = None) resistics.letsgo.ResisticsEnvironment[source]¶
Load an existing project into a ResisticsEnvironment
- Parameters
dir_path (Union[Path, str]) – The project directory
config (Optional[Configuration], optional) – A configuration of parameters to use
- Returns
The ResisticsEnvironment combining a project and a configuration
- Return type
- Raises
ProjectLoadError – If the loading failed
- resistics.letsgo.reload(resenv: resistics.letsgo.ResisticsEnvironment) resistics.letsgo.ResisticsEnvironment[source]¶
Reload the project in the ResisticsEnvironment
- Parameters
resenv (ResisticsEnvironment) – The current resistics environment
- Returns
The resistics environment with the project reloaded
- Return type
- resistics.letsgo.run_time_processors(config: resistics.config.Configuration, time_data: resistics.time.TimeData) resistics.time.TimeData[source]¶
Process time data
- Parameters
config (Configuration) – The configuration
time_data (TimeData) – Time data to process
- Returns
Process time data
- Return type
- resistics.letsgo.run_decimation(config: resistics.config.Configuration, time_data: resistics.time.TimeData, dec_params: Optional[resistics.decimate.DecimationParameters] = None) resistics.decimate.DecimatedData[source]¶
Decimate TimeData
- Parameters
config (Configuration) – The configuration
time_data (TimeData) – Time data to decimate
dec_params (DecimationParameters) – Number of levels, decimation factors etc.
- Returns
Decimated time data
- Return type
- resistics.letsgo.run_windowing(config: resistics.config.Configuration, ref_time: resistics.sampling.HighResDateTime, dec_data: resistics.decimate.DecimatedData) resistics.window.WindowedData[source]¶
Window time data
- Parameters
config (Configuration) – The configuration
ref_time (HighResDateTime) – The reference time
dec_data (DecimatedData) – Decimated data to window
- Returns
The windowed data
- Return type
- resistics.letsgo.run_fft(config: resistics.config.Configuration, win_data: resistics.window.WindowedData) resistics.spectra.SpectraData[source]¶
Run Fourier transform
- Parameters
config (Configuration) – The configuration
win_data (WindowedData) – Windowed data
- Returns
Fourier transformed windowed data
- Return type
- resistics.letsgo.run_spectra_processors(config: resistics.config.Configuration, spec_data: resistics.spectra.SpectraData) resistics.spectra.SpectraData[source]¶
Run any spectra processors
- Parameters
config (Configuration) – The configuration
spec_data (SpectraData) – Spectra data
- Returns
Processed spectra data
- Return type
- resistics.letsgo.run_evals(config: resistics.config.Configuration, dec_params: resistics.decimate.DecimationParameters, spec_data: resistics.spectra.SpectraData) resistics.spectra.SpectraData[source]¶
Run evaluation frequency data calculator
- Parameters
config (Configuration) – The configuration
dec_params (DecimationParameters) – Decimation parameters with the evaluation frequencies
spec_data (SpectraData) – The spectra data
- Returns
Spectra data at evaluation frequencies
- Return type
- resistics.letsgo.run_sensor_calibration(config: resistics.config.Configuration, calibration_path: pathlib.Path, spec_data: resistics.spectra.SpectraData) resistics.spectra.SpectraData[source]¶
Run calibration
- Parameters
config (Configuration) – The configuration
calibration_path (Path) – Path to calibration data
spec_data (SpectraData) – Spectra data to calibrate
- Returns
Calibrated spectra data
- Return type
- resistics.letsgo.run_regression_preparer(config: resistics.config.Configuration, gathered_data: resistics.gather.GatheredData) resistics.regression.RegressionInputData[source]¶
Prepare linear regression data
- Parameters
config (Configuration) – The configuration
gathered_data (GatheredData) – Gathered data to input into the regression
- Returns
Regression inputs for all evaluation frequencies
- Return type
- resistics.letsgo.run_solver(config: resistics.config.Configuration, reg_data: resistics.regression.RegressionInputData) resistics.regression.Solution[source]¶
Run the regression solver
- Parameters
config (Configuration) – The configuration
reg_data (RegressionInputData) – The regression input data
- Returns
Transfer function estimate
- Return type
- resistics.letsgo.quick_read(dir_path: pathlib.Path, config: Optional[resistics.config.Configuration] = None) resistics.time.TimeData[source]¶
Read time data folder
- Parameters
dir_path (Path) – The directory path to read
config (Optional[Configuration], optional) – Configuration with appropriate readers, by default None.
- Returns
The read TimeData
- Return type
- Raises
TimeDataReadError – If unable to read data
- resistics.letsgo.quick_view(dir_path: pathlib.Path, config: Optional[resistics.config.Configuration] = None, decimate: bool = False, max_pts: int = 10000)[source]¶
Quick plotting of time data
- Parameters
dir_path (Path) – The directory path
config (Optional[Configuration], optional) – The configuration with the required time readers, by default None
decimate (bool, optional) – Boolean flag for decimating, by default False
max_pts (Optional[int], optional) – Max points in lttb decimation, by default 10_000
- Returns
Plotly figure
- Return type
go.Figure
- Raises
ValueError – If time data fails reading
- resistics.letsgo.quick_spectra(dir_path: pathlib.Path, config: Optional[resistics.config.Configuration] = None) resistics.spectra.SpectraData[source]¶
Quick plotting of time data
- Parameters
dir_path (Path) – The directory path
config (Optional[Configuration], optional) – The configuration with the required time readers, by default None
- Returns
The spectra data
- Return type
- Raises
ValueError – If time data fails reading
- resistics.letsgo.quick_tf(dir_path: pathlib.Path, config: Optional[resistics.config.Configuration] = None, calibration_path: Optional[pathlib.Path] = None) resistics.regression.Solution[source]¶
Quickly calculate out a transfer function for time data in its own directory
- Parameters
dir_path (Path) – The directory path
config (Optional[Configuration], optional) – A configuration instance, by default None
calibration_path (Optional[Path], optional) – The path to the calibration data, by default None
- Returns
Transfer function estimate
- Return type
- resistics.letsgo.process_time(resenv: resistics.letsgo.ResisticsEnvironment, site_name: str, meas_name: str, out_site: str, out_meas: str, from_time: Optional[Union[str, pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime]] = None, to_time: Optional[Union[str, pandas._libs.tslibs.timestamps.Timestamp, datetime.datetime]] = None) None[source]¶
Process time data and save as a new measurement
This is useful when resampling data to use with other measurements
- Parameters
resenv (ResisticsEnvironment) – The resistics environment
site_name (str) – The name of the site with the data to process
meas_name (str) – The name of the measurement to process
out_site (str) – The site to output the data to
out_meas (str) – The name of the measurement to output the data to
from_time (Optional[DateTimeLike], optional) – Time to get the time data from, by default None
to_time (Optional[DateTimeLike], optional) – Time to get the time data up to, by default None
- resistics.letsgo.process_time_to_evals(resenv: resistics.letsgo.ResisticsEnvironment, site_name: str, meas_name: str) None[source]¶
Process from time data to Fourier spectra
- Parameters
resenv (ResisticsEnvironment) – The resistics environment containing the project and configuration
site_name (str) – The name of the site
meas_name (str) – The name of the measurement to process
- resistics.letsgo.process_evals_to_tf(resenv: resistics.letsgo.ResisticsEnvironment, fs: float, out_site: str, in_site: Optional[str] = None, cross_site: Optional[str] = None, masks: Optional[Dict[str, str]] = None, postfix: Optional[str] = None) resistics.regression.Solution[source]¶
Process spectra to transfer functions
- Parameters
resenv (ResisticsEnvironment) – The resistics environment
fs (float) – The sampling frequency to process
out_site (str) – The name of the output site
in_site (Optional[str], optional) – The name of the input site, by default None. This should be used for intersite processing
cross_site (Optional[str], optional) – The name of the cross site, by default None. This is usually the site to use as the remote reference.
masks (Optional[Dict[str, str]], optional) – Any masks to apply, by default None
postfix (Optional[str]) – String to add to the end of solution, by default None
- Returns
Transfer function estimate
- Return type
- resistics.letsgo.get_solution(resenv: resistics.letsgo.ResisticsEnvironment, site_name: str, config_name: str, fs: float, tf_name: str, tf_var: str, postfix: Optional[str] = None) resistics.regression.Solution[source]¶
Get a solution
- Parameters
resenv (ResisticsEnvironment) – The resistics environment
site_name (str) – The site for which to get the solution
config_name (str) – The configuration that was used
fs (float) – The sampling frequency
tf_name (str) – The transfer function name
tf_var (str) – The transfer function variation
postfix (Optional[str], optional) – Any postfix on the solution, by default None
- Returns
The solution
- Return type