resistics.window module

Module for calculating window related data. Windows can be indexed relative to two starting indices.

  • Local window index

    • Window index relative to the TimeData is called “local_win”

    • Local window indices always start at 0

  • Global window index

    • The global window index is relative to the project reference time

    • The 0 index window begins at the reference time

    • This window indexing is to synchronise data across sites

The global window index is considered the default and sometimes referred to as the window. Local windows should be explicitly referred to as local_win in all cases.

The window module includes functionality to do the following:

  • Windowing utility functions to calculate window and overlap sizes

  • Functions to map windows to samples in TimeData

  • Converting a global index array to datetime

Usually with windowing, there is a window size and windows overlap with each other for a set number of samples. As an illustrative examples, consider a signal sampled at 10 Hz (dt=0.1 seconds) with 24 samples. This will be windowed using a window size of 8 samples per window and a 2 sample overlap.

>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> fs = 10
>>> n_samples = 24
>>> win_size = 8
>>> olap_size = 2
>>> times = np.arange(0, n_samples) * (1/fs)

The first window

>>> start_win1 = 0
>>> end_win1 = win_size
>>> win1_times = times[start_win1:end_win1]

The second window

>>> start_win2 = end_win1 - olap_size
>>> end_win2 = start_win2 + win_size
>>> win2_times = times[start_win2:end_win2]

The third window

>>> start_win3 = end_win2 - olap_size
>>> end_win3 = start_win3 + win_size
>>> win3_times = times[start_win3:end_win3]

The fourth window

>>> start_win4= end_win3 - olap_size
>>> end_win4 = start_win4 + win_size
>>> win4_times = times[start_win4:end_win4]

Let’s look at the actual window times for each window

>>> win1_times
array([0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7])
>>> win2_times
array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3])
>>> win3_times
array([1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9])
>>> win4_times
array([1.8, 1.9, 2. , 2.1, 2.2, 2.3])

The duration and increments of windows can be calculated using provided methods

>>> from resistics.window import win_duration, inc_duration
>>> print(win_duration(win_size, fs))
0:00:00.7
>>> print(inc_duration(win_size, olap_size, fs))
0:00:00.6

Plot the windows to give an illustration of how it works

>>> plt.plot(win1_times, np.ones_like(win1_times), "bo", label="window1") 
>>> plt.plot(win2_times, np.ones_like(win2_times)*2, "ro", label="window2") 
>>> plt.plot(win3_times, np.ones_like(win3_times)*3, "go", label="window3") 
>>> plt.plot(win4_times, np.ones_like(win4_times)*4, "co", label="window4") 
>>> plt.xlabel("Time [s]") 
>>> plt.legend() 
>>> plt.grid() 
>>> plt.tight_layout() 
>>> plt.show()

(Source code, png, hires.png, pdf)

_images/resistics-window-1.png
resistics.window.win_duration(win_size: int, fs: float) → attotime.objects.attotimedelta.attotimedelta[source]

Get the window duration

Parameters

  • win_size (int) – Window size in samples

  • fs (float) – Sampling frequency Hz

Returns

Duration

Return type

RSTimeDelta

Examples

A few examples with different sampling frequencies and window sizes

>>> from resistics.window import win_duration
>>> duration = win_duration(512, 512)
>>> print(duration)
0:00:00.998046875
>>> duration = win_duration(520, 512)
>>> print(duration)
0:00:01.013671875
>>> duration = win_duration(4096, 16_384)
>>> print(duration)
0:00:00.24993896484375
>>> duration = win_duration(200, 0.05)
>>> print(duration)
1:06:20
resistics.window.inc_duration(win_size: int, olap_size: int, fs: float) → attotime.objects.attotimedelta.attotimedelta[source]

Get the increment between window start times

If the overlap size = 0, then the time increment between windows is simply the window duration. However, when there is an overlap, the increment between window start times has to be adjusted by the overlap size

Parameters

  • win_size (int) – The window size in samples

  • olap_size (int) – The overlap size in samples

  • fs (float) – The sample frequency Hz

Returns

The duration of the window

Return type

RSTimeDelta

Examples

>>> from resistics.window import inc_duration
>>> increment = inc_duration(128, 32, 128)
>>> print(increment)
0:00:00.75
>>> increment = inc_duration(128*3600, 128*60, 128)
>>> print(increment)
0:59:00
resistics.window.win_to_datetime(ref_time: attotime.objects.attodatetime.attodatetime, global_win: int, increment: attotime.objects.attotimedelta.attotimedelta) → attotime.objects.attodatetime.attodatetime[source]

Convert reference window index to start time of window

Parameters

  • ref_time (RSDateTime) – Reference time

  • global_win (int) – Window index relative to reference time

  • increment (RSTimeDelta) – The increment duration

Returns

Start time of window

Return type

RSDateTime

Examples

An example with sampling at 1 Hz, a window size of 100 and an overlap size of 25.

>>> from resistics.sampling import to_datetime
>>> from resistics.window import inc_duration, win_to_datetime
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> fs = 1
>>> win_size = 60
>>> olap_size = 15
>>> increment = inc_duration(win_size, olap_size, fs)
>>> print(increment)
0:00:45

The increment is the time increment between the start of time one window and the succeeding window.

>>> print(win_to_datetime(ref_time, 0, increment))
2021-01-01 00:00:00
>>> print(win_to_datetime(ref_time, 1, increment))
2021-01-01 00:00:45
>>> print(win_to_datetime(ref_time, 2, increment))
2021-01-01 00:01:30
>>> print(win_to_datetime(ref_time, 3, increment))
2021-01-01 00:02:15
resistics.window.datetime_to_win(ref_time: attotime.objects.attodatetime.attodatetime, time: attotime.objects.attodatetime.attodatetime, increment: attotime.objects.attotimedelta.attotimedelta, method: str = 'round') → int[source]

Convert a datetime to a global window index

Parameters

  • ref_time (RSDateTime) – Reference time

  • time (RSDateTime) – Datetime to convert

  • increment (RSTimeDelta) – The increment duration

  • method (str, optional) – Method for dealing with float results, by default “round”

Returns

The global window index i.e. the window index relative to the reference time

Return type

int

Raises

ValueError – If time < ref_time

Examples

A simple example to show the logic

>>> from resistics.sampling import to_datetime, to_timedelta
>>> from resistics.window import datetime_to_win, win_to_datetime, inc_duration
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> time = to_datetime("2021-01-01 00:01:00")
>>> increment = to_timedelta(60)
>>> global_win = datetime_to_win(ref_time, time, increment)
>>> global_win
1
>>> print(win_to_datetime(ref_time, global_win, increment))
2021-01-01 00:01:00

A more complex logic with window sizes, overlap sizes and sampling frequencies

>>> fs = 128
>>> win_size = 256
>>> olap_size = 64
>>> ref_time = to_datetime("2021-03-15 00:00:00")
>>> time = to_datetime("2021-04-17 18:00:00")
>>> increment = inc_duration(win_size, olap_size, fs)
>>> print(increment)
0:00:01.5
>>> global_win = datetime_to_win(ref_time, time, increment)
>>> global_win
1944000
>>> print(win_to_datetime(ref_time, global_win, increment))
2021-04-17 18:00:00

In this scenario, explore the use of rounding

>>> time = to_datetime("2021-04-17 18:00:00.50")
>>> global_win = datetime_to_win(ref_time, time, increment, method = "floor")
>>> global_win
1944000
>>> print(win_to_datetime(ref_time, global_win, increment))
2021-04-17 18:00:00
>>> global_win = datetime_to_win(ref_time, time, increment, method = "ceil")
>>> global_win
1944001
>>> print(win_to_datetime(ref_time, global_win, increment))
2021-04-17 18:00:01.5
>>> global_win = datetime_to_win(ref_time, time, increment, method = "round")
>>> global_win
1944000
>>> print(win_to_datetime(ref_time, global_win, increment))
2021-04-17 18:00:00

Another example with a window duration of greater than a day

>>> fs = 4.8828125e-05
>>> win_size = 64
>>> olap_size = 16
>>> ref_time = to_datetime("1985-07-18 01:00:20")
>>> time = to_datetime("1985-09-22 23:00:00")
>>> increment = inc_duration(win_size, olap_size, fs)
>>> print(increment)
11 days, 9:04:00
>>> global_win = datetime_to_win(ref_time, time, increment)
>>> global_win
6
>>> print(win_to_datetime(ref_time, global_win, increment))
1985-09-24 07:24:20

This time is greater than the time that was transformed to global window, 1985-09-22 23:00:00. Try again, this time with the floor option.

>>> global_win = datetime_to_win(ref_time, time, increment, method="floor")
>>> global_win
5
>>> print(win_to_datetime(ref_time, global_win, increment))
1985-09-12 22:20:20
resistics.window.get_first_and_last_win(ref_time: attotime.objects.attodatetime.attodatetime, metadata: resistics.decimate.DecimatedLevelMetadata, win_size: int, olap_size: int) → Tuple[int, int][source]

Get first and last window for a decimated data level

Note

For the last window, on initial calculation this may be one or a maximum of two windows beyond the last time. The last window is adjusted in this function.

Two windows may occur when the time of the last sample is in the overlap of the final two windows.

Parameters

  • ref_time (RSDateTime) – The reference time

  • metadata (DecimatedLevelMetadata) – Metadata for the decimation level

  • win_size (int) – Window size in samples

  • olap_size (int) – Overlap size in samples

Returns

First and last global windows. This is window indices relative to the reference time

Return type

Tuple[int, int]

Raises

ValueError – If unable to calculate the last window correctly as this will result in an incorrect number of windows

Examples

Get the first and last window for the first decimation level in a decimated data instance.

>>> from resistics.testing import decimated_data_random
>>> from resistics.sampling import to_datetime
>>> from resistics.window import get_first_and_last_win, win_to_datetime
>>> from resistics.window import win_duration, inc_duration
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> dec_data = decimated_data_random(fs=0.1, first_time="2021-01-01 00:05:10", n_samples=100, factor=10)

Get the metadata for decimation level 0

>>> level_metadata = dec_data.metadata.levels_metadata[0]
>>> level_metadata.summary()
{
    'fs': 10.0,
    'n_samples': 10000,
    'first_time': '2021-01-01 00:05:10.000000_000000_000000_000000',
    'last_time': '2021-01-01 00:21:49.899999_999999_977300_000000'
}

Note

As a point of interest, note how the last time is actually slightly incorrect. This is due to machine precision issues described in more detail here https://docs.python.org/3/tutorial/floatingpoint.html. Whilst there is value in using the high resolution datetime format for high sampling rates, there is a tradeoff. Such are the perils of floating point arithmetic.

The next step is to calculate the first and last window, relative to the reference time

>>> win_size = 100
>>> olap_size = 25
>>> first_win, last_win = get_first_and_last_win(ref_time, level_metadata, win_size, olap_size)
>>> print(first_win, last_win)
42 173

These window indices can be converted to start times of the windows. The last window is checked to make sure it does not extend past the end of the time data. First get the window duration and increments.

>>> duration = win_duration(win_size, level_metadata.fs)
>>> print(duration)
0:00:09.9
>>> increment = inc_duration(win_size, olap_size, level_metadata.fs)
>>> print(increment)
0:00:07.5

Now calculate the times of the windows

>>> first_win_start_time = win_to_datetime(ref_time, 42, increment)
>>> last_win_start_time = win_to_datetime(ref_time, 173, increment)
>>> print(first_win_start_time, last_win_start_time)
2021-01-01 00:05:15 2021-01-01 00:21:37.5
>>> print(last_win_start_time + duration)
2021-01-01 00:21:47.4
>>> print(level_metadata.last_time)
2021-01-01 00:21:49.8999999999999773
>>> level_metadata.last_time > last_win_start_time + increment
True
resistics.window.get_win_starts(ref_time: attotime.objects.attodatetime.attodatetime, win_size: int, olap_size: int, fs: float, n_wins: int, index_offset: int) → pandas.core.indexes.datetimes.DatetimeIndex[source]

Get window start times

This is a useful for getting the timestamps for the windows in a dataset

Parameters

  • ref_time (RSDateTime) – The reference time

  • win_size (int) – The window size

  • olap_size (int) – The overlap size

  • fs (float) – The sampling frequency

  • n_wins (int) – The number of windows

  • index_offset (int) – The index offset from the reference time

Returns

The start times of the windows

Return type

pd.DatetimeIndex

Examples

>>> import pandas as pd
>>> from resistics.sampling import to_datetime
>>> from resistics.window import get_win_starts
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> win_size = 100
>>> olap_size = 25
>>> fs = 10
>>> n_wins = 3
>>> index_offset = 480
>>> starts = get_win_starts(ref_time, win_size, olap_size, fs, n_wins, index_offset)
>>> pd.Series(starts)
0   2021-01-01 01:00:00.000
1   2021-01-01 01:00:07.500
2   2021-01-01 01:00:15.000
dtype: datetime64[ns]
resistics.window.get_win_ends(starts: pandas.core.indexes.datetimes.DatetimeIndex, win_size: int, fs: float) → pandas.core.indexes.datetimes.DatetimeIndex[source]

Get window end times

Parameters

  • starts (RSDateTime) – The start times of the windows

  • win_size (int) – The window size

  • fs (float) – The sampling frequency

Returns

The end times of the windows

Return type

pd.DatetimeIndex

Examples

>>> import pandas as pd
>>> from resistics.sampling import to_datetime
>>> from resistics.window import get_win_starts, get_win_ends
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> win_size = 100
>>> olap_size = 25
>>> fs = 10
>>> n_wins = 3
>>> index_offset = 480
>>> starts = get_win_starts(ref_time, win_size, olap_size, fs, n_wins, index_offset)
>>> pd.Series(starts)
0   2021-01-01 01:00:00.000
1   2021-01-01 01:00:07.500
2   2021-01-01 01:00:15.000
dtype: datetime64[ns]
>>> ends = get_win_ends(starts, win_size, fs)
>>> pd.Series(ends)
0   2021-01-01 01:00:09.900
1   2021-01-01 01:00:17.400
2   2021-01-01 01:00:24.900
dtype: datetime64[ns]
resistics.window.get_win_table(ref_time: attotime.objects.attodatetime.attodatetime, metadata: resistics.decimate.DecimatedLevelMetadata, win_size: int, olap_size: int) → pandas.core.frame.DataFrame[source]

Get a DataFrame with

Parameters

  • ref_time (RSDateTime) – Reference

  • metadata (DecimatedLevelMetadata) – Metadata for the decimation level

  • win_size (int) – The window size

  • olap_size (int) – The overlap size

Returns

A pandas DataFrame with details about each window

Return type

pd.DataFrame

Examples

>>> import matplotlib.pyplot as plt
>>> from resistics.decimate import DecimatedLevelMetadata
>>> from resistics.sampling import to_datetime, to_timedelta
>>> from resistics.window import get_win_table
>>> ref_time = to_datetime("2021-01-01 00:00:00")
>>> fs = 10
>>> n_samples = 1000
>>> first_time = to_datetime("2021-01-01 01:00:00")
>>> last_time = first_time + to_timedelta((n_samples-1)/fs)
>>> metadata = DecimatedLevelMetadata(fs=10, n_samples=1000, first_time=first_time, last_time=last_time)
>>> print(metadata.fs, metadata.first_time, metadata.last_time)
10.0 2021-01-01 01:00:00 2021-01-01 01:01:39.9
>>> win_size = 100
>>> olap_size = 25
>>> df = get_win_table(ref_time, metadata, win_size, olap_size)
>>> print(df.to_string())
    global  local  from_sample  to_sample               win_start                 win_end
0      480      0            0         99 2021-01-01 01:00:00.000 2021-01-01 01:00:09.900
1      481      1           75        174 2021-01-01 01:00:07.500 2021-01-01 01:00:17.400
2      482      2          150        249 2021-01-01 01:00:15.000 2021-01-01 01:00:24.900
3      483      3          225        324 2021-01-01 01:00:22.500 2021-01-01 01:00:32.400
4      484      4          300        399 2021-01-01 01:00:30.000 2021-01-01 01:00:39.900
5      485      5          375        474 2021-01-01 01:00:37.500 2021-01-01 01:00:47.400
6      486      6          450        549 2021-01-01 01:00:45.000 2021-01-01 01:00:54.900
7      487      7          525        624 2021-01-01 01:00:52.500 2021-01-01 01:01:02.400
8      488      8          600        699 2021-01-01 01:01:00.000 2021-01-01 01:01:09.900
9      489      9          675        774 2021-01-01 01:01:07.500 2021-01-01 01:01:17.400
10     490     10          750        849 2021-01-01 01:01:15.000 2021-01-01 01:01:24.900
11     491     11          825        924 2021-01-01 01:01:22.500 2021-01-01 01:01:32.400
12     492     12          900        999 2021-01-01 01:01:30.000 2021-01-01 01:01:39.900

Plot six windows to illustrate the overlap

>>> plt.figure(figsize=(8, 3)) 
>>> for idx, row in df.iterrows():
...     color = "red" if idx%2 == 0 else "blue"
...     plt.axvspan(row.loc["win_start"], row.loc["win_end"], alpha=0.5, color=color) 
...     if idx > 5:
...         break
>>> plt.tight_layout() 
>>> plt.show()

(Source code, png, hires.png, pdf)

_images/resistics-window-2.png
pydantic model resistics.window.WindowParameters[source]

Bases: resistics.common.ResisticsModel

Windowing parameters per decimation level

Windowing parameters are the window and overlap size for each decimation level.

Parameters

  • n_levels (int) – The number of decimation levels

  • min_n_wins (int) – Minimum number of windows

  • win_sizes (List[int]) – The window sizes per decimation level

  • olap_sizes (List[int]) – The overlap sizes per decimation level

Examples

Generate decimation and windowing parameters for data sampled at 4096 Hz. Note that requesting window sizes or overlap sizes for decimation levels that do not exist will raise a ValueError.

>>> from resistics.decimate import DecimationSetup
>>> from resistics.window import WindowSetup
>>> dec_setup = DecimationSetup(n_levels=3, per_level=3)
>>> dec_params = dec_setup.run(4096)
>>> dec_params.summary()
{
    'fs': 4096.0,
    'n_levels': 3,
    'per_level': 3,
    'min_samples': 256,
    'eval_freqs': [
        1024.0,
        724.0773439350246,
        512.0,
        362.0386719675123,
        256.0,
        181.01933598375615,
        128.0,
        90.50966799187808,
        64.0
    ],
    'dec_factors': [1, 2, 8],
    'dec_increments': [1, 2, 4],
    'dec_fs': [4096.0, 2048.0, 512.0]
}
>>> win_params = WindowSetup().run(dec_params.n_levels, dec_params.dec_fs)
>>> win_params.summary()
{
    'n_levels': 3,
    'min_n_wins': 5,
    'win_sizes': [1024, 512, 128],
    'olap_sizes': [256, 128, 32]
}
>>> win_params.get_win_size(0)
1024
>>> win_params.get_olap_size(0)
256
>>> win_params.get_olap_size(3)
Traceback (most recent call last):
...
ValueError: Level 3 must be 0 <= level < 3

Show JSON schema
{
   "title": "WindowParameters",
   "description": "Windowing parameters per decimation level\n\nWindowing parameters are the window and overlap size for each decimation\nlevel.\n\nParameters\n----------\nn_levels : int\n    The number of decimation levels\nmin_n_wins : int\n    Minimum number of windows\nwin_sizes : List[int]\n    The window sizes per decimation level\nolap_sizes : List[int]\n    The overlap sizes per decimation level\n\nExamples\n--------\nGenerate decimation and windowing parameters for data sampled at 4096 Hz.\nNote that requesting window sizes or overlap sizes for decimation levels\nthat do not exist will raise a ValueError.\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.window import WindowSetup\n>>> dec_setup = DecimationSetup(n_levels=3, per_level=3)\n>>> dec_params = dec_setup.run(4096)\n>>> dec_params.summary()\n{\n    'fs': 4096.0,\n    'n_levels': 3,\n    'per_level': 3,\n    'min_samples': 256,\n    'eval_freqs': [\n        1024.0,\n        724.0773439350246,\n        512.0,\n        362.0386719675123,\n        256.0,\n        181.01933598375615,\n        128.0,\n        90.50966799187808,\n        64.0\n    ],\n    'dec_factors': [1, 2, 8],\n    'dec_increments': [1, 2, 4],\n    'dec_fs': [4096.0, 2048.0, 512.0]\n}\n>>> win_params = WindowSetup().run(dec_params.n_levels, dec_params.dec_fs)\n>>> win_params.summary()\n{\n    'n_levels': 3,\n    'min_n_wins': 5,\n    'win_sizes': [1024, 512, 128],\n    'olap_sizes': [256, 128, 32]\n}\n>>> win_params.get_win_size(0)\n1024\n>>> win_params.get_olap_size(0)\n256\n>>> win_params.get_olap_size(3)\nTraceback (most recent call last):\n...\nValueError: Level 3 must be 0 <= level < 3",
   "type": "object",
   "properties": {
      "n_levels": {
         "title": "N Levels",
         "type": "integer"
      },
      "min_n_wins": {
         "title": "Min N Wins",
         "type": "integer"
      },
      "win_sizes": {
         "title": "Win Sizes",
         "type": "array",
         "items": {
            "type": "integer"
         }
      },
      "olap_sizes": {
         "title": "Olap Sizes",
         "type": "array",
         "items": {
            "type": "integer"
         }
      }
   },
   "required": [
      "n_levels",
      "min_n_wins",
      "win_sizes",
      "olap_sizes"
   ]
}

field n_levels: int [Required]
field min_n_wins: int [Required]
field win_sizes: List[int] [Required]
field olap_sizes: List[int] [Required]
check_level(level: int)[source]

Check the decimation level is within range

get_win_size(level: int) → int[source]

Get window size for a decimation level

get_olap_size(level: int) → int[source]

Get overlap size for a decimation level

pydantic model resistics.window.WindowSetup[source]

Bases: resistics.common.ResisticsProcess

Setup WindowParameters

WindowSetup outputs the WindowParameters to use for windowing decimated time data.

Window parameters are simply the window and overlap sizes for each decimation level.

Parameters

  • min_size (int, optional) – Minimum window size, by default 128

  • min_olap (int, optional) – Minimum overlap size, by default 32

  • win_factor (int, optional) – Window factor, by default 4. Window sizes are calculated by sampling frequency / 4 to ensure sufficient frequency resolution. If the sampling frequency is small, window size will be adjusted to min_size

  • olap_proportion (float, optional) – The proportion of the window size to use as the overlap, by default 0.25. For example, for a window size of 128, the overlap would be 0.25 * 128 = 32

  • min_n_wins (int, optional) – The minimum number of windows needed in a decimation level, by default 5

  • win_sizes (Optional[List[int]], optional) – Explicit define window sizes, by default None. Must have the same length as number of decimation levels

  • olap_sizes (Optional[List[int]], optional) – Explicitly define overlap sizes, by default None. Must have the same length as number of decimation levels

Examples

Generate decimation and windowing parameters for data sampled at 0.05 Hz or 20 seconds sampling period

>>> from resistics.decimate import DecimationSetup
>>> from resistics.window import WindowSetup
>>> dec_params = DecimationSetup(n_levels=3, per_level=3).run(0.05)
>>> dec_params.summary()
{
    'fs': 0.05,
    'n_levels': 3,
    'per_level': 3,
    'min_samples': 256,
    'eval_freqs': [
        0.0125,
        0.008838834764831844,
        0.00625,
        0.004419417382415922,
        0.003125,
        0.002209708691207961,
        0.0015625,
        0.0011048543456039805,
        0.00078125
    ],
    'dec_factors': [1, 2, 8],
    'dec_increments': [1, 2, 4],
    'dec_fs': [0.05, 0.025, 0.00625]
}
>>> win_params = WindowSetup().run(dec_params.n_levels, dec_params.dec_fs)
>>> win_params.summary()
{
    'n_levels': 3,
    'min_n_wins': 5,
    'win_sizes': [128, 128, 128],
    'olap_sizes': [32, 32, 32]
}

Window parameters can also be explicitly defined

>>> from resistics.decimate import DecimationSetup
>>> from resistics.window import WindowSetup
>>> dec_setup = DecimationSetup(n_levels=3, per_level=3)
>>> dec_params = dec_setup.run(0.05)
>>> win_setup = WindowSetup(win_sizes=[1000, 578, 104])
>>> win_params = win_setup.run(dec_params.n_levels, dec_params.dec_fs)
>>> win_params.summary()
{
    'n_levels': 3,
    'min_n_wins': 5,
    'win_sizes': [1000, 578, 104],
    'olap_sizes': [250, 144, 32]
}

Show JSON schema
{
   "title": "WindowSetup",
   "description": "Setup WindowParameters\n\nWindowSetup outputs the WindowParameters to use for windowing decimated\ntime data.\n\nWindow parameters are simply the window and overlap sizes for each\ndecimation level.\n\nParameters\n----------\nmin_size : int, optional\n    Minimum window size, by default 128\nmin_olap : int, optional\n    Minimum overlap size, by default 32\nwin_factor : int, optional\n    Window factor, by default 4. Window sizes are calculated by sampling\n    frequency / 4 to ensure sufficient frequency resolution. If the\n    sampling frequency is small, window size will be adjusted to\n    min_size\nolap_proportion : float, optional\n    The proportion of the window size to use as the overlap, by default\n    0.25. For example, for a window size of 128, the overlap would be\n    0.25 * 128 = 32\nmin_n_wins : int, optional\n    The minimum number of windows needed in a decimation level, by\n    default 5\nwin_sizes : Optional[List[int]], optional\n    Explicit define window sizes, by default None. Must have the same\n    length as number of decimation levels\nolap_sizes : Optional[List[int]], optional\n    Explicitly define overlap sizes, by default None. Must have the same\n    length as number of decimation levels\n\nExamples\n--------\nGenerate decimation and windowing parameters for data sampled at 0.05 Hz or\n20 seconds sampling period\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.window import WindowSetup\n>>> dec_params = DecimationSetup(n_levels=3, per_level=3).run(0.05)\n>>> dec_params.summary()\n{\n    'fs': 0.05,\n    'n_levels': 3,\n    'per_level': 3,\n    'min_samples': 256,\n    'eval_freqs': [\n        0.0125,\n        0.008838834764831844,\n        0.00625,\n        0.004419417382415922,\n        0.003125,\n        0.002209708691207961,\n        0.0015625,\n        0.0011048543456039805,\n        0.00078125\n    ],\n    'dec_factors': [1, 2, 8],\n    'dec_increments': [1, 2, 4],\n    'dec_fs': [0.05, 0.025, 0.00625]\n}\n>>> win_params = WindowSetup().run(dec_params.n_levels, dec_params.dec_fs)\n>>> win_params.summary()\n{\n    'n_levels': 3,\n    'min_n_wins': 5,\n    'win_sizes': [128, 128, 128],\n    'olap_sizes': [32, 32, 32]\n}\n\nWindow parameters can also be explicitly defined\n\n>>> from resistics.decimate import DecimationSetup\n>>> from resistics.window import WindowSetup\n>>> dec_setup = DecimationSetup(n_levels=3, per_level=3)\n>>> dec_params = dec_setup.run(0.05)\n>>> win_setup = WindowSetup(win_sizes=[1000, 578, 104])\n>>> win_params = win_setup.run(dec_params.n_levels, dec_params.dec_fs)\n>>> win_params.summary()\n{\n    'n_levels': 3,\n    'min_n_wins': 5,\n    'win_sizes': [1000, 578, 104],\n    'olap_sizes': [250, 144, 32]\n}",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "min_size": {
         "title": "Min Size",
         "default": 128,
         "type": "integer"
      },
      "min_olap": {
         "title": "Min Olap",
         "default": 32,
         "type": "integer"
      },
      "win_factor": {
         "title": "Win Factor",
         "default": 4,
         "type": "integer"
      },
      "olap_proportion": {
         "title": "Olap Proportion",
         "default": 0.25,
         "type": "number"
      },
      "min_n_wins": {
         "title": "Min N Wins",
         "default": 5,
         "type": "integer"
      },
      "win_sizes": {
         "title": "Win Sizes",
         "type": "array",
         "items": {
            "type": "integer"
         }
      },
      "olap_sizes": {
         "title": "Olap Sizes",
         "type": "array",
         "items": {
            "type": "integer"
         }
      }
   }
}

field min_size: int = 128
field min_olap: int = 32
field win_factor: int = 4
field olap_proportion: float = 0.25
field min_n_wins: int = 5
field win_sizes: Optional[List[int]] = None
field olap_sizes: Optional[List[int]] = None
run(n_levels: int, dec_fs: List[float]) → resistics.window.WindowParameters[source]

Calculate window and overlap sizes for each decimation level based on decimation level sampling frequency and minimum allowable parameters

The window and overlap sizes (number of samples) are calculated based in the following way:

  • window size = frequency at decimation level / window factor

  • overlap size = window size * overlap proportion

This is to ensure good frequency resolution at high frequencies. At low sampling frequencies, this would result in very small window sizes, therefore, there a minimum allowable sizes for both windows and overlap defined by min_size and min_olap in the initialiser. If window sizes or overlaps size are calculated below these respecitively, they will be set to the minimum values.

Parameters

  • n_levels (int) – The number of decimation levels

  • dec_fs (List[float]) – The sampling frequencies for each decimation level

Returns

The window parameters, the window sizes and overlaps for each decimation level

Return type

WindowParameters

Raises

  • ValueError – If the number of windows does not match the number of levels

  • ValueError – If the number of overlaps does not match the number of levels

pydantic model resistics.window.WindowedLevelMetadata[source]

Bases: resistics.common.Metadata

Metadata for a windowed level

Show JSON schema
{
   "title": "WindowedLevelMetadata",
   "description": "Metadata for a windowed level",
   "type": "object",
   "properties": {
      "fs": {
         "title": "Fs",
         "type": "number"
      },
      "n_wins": {
         "title": "N Wins",
         "type": "integer"
      },
      "win_size": {
         "title": "Win Size",
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "olap_size": {
         "title": "Olap Size",
         "exclusiveMinimum": 0,
         "type": "integer"
      },
      "index_offset": {
         "title": "Index Offset",
         "type": "integer"
      }
   },
   "required": [
      "fs",
      "n_wins",
      "win_size",
      "olap_size",
      "index_offset"
   ]
}

field fs: float [Required]

The sampling frequency for the decimation level

field n_wins: int [Required]

The number of windows

field win_size: pydantic.types.PositiveInt [Required]

The window size in samples

Constraints

  • exclusiveMinimum = 0

field olap_size: pydantic.types.PositiveInt [Required]

The overlap size in samples

Constraints

  • exclusiveMinimum = 0

field index_offset: int [Required]

The global window offset for local window 0

property dt
pydantic model resistics.window.WindowedMetadata[source]

Bases: resistics.common.WriteableMetadata

Metadata for windowed data

Show JSON schema
{
   "title": "WindowedMetadata",
   "description": "Metadata for windowed data",
   "type": "object",
   "properties": {
      "file_info": {
         "$ref": "#/definitions/ResisticsFile"
      },
      "fs": {
         "title": "Fs",
         "type": "array",
         "items": {
            "type": "number"
         }
      },
      "chans": {
         "title": "Chans",
         "type": "array",
         "items": {
            "type": "string"
         }
      },
      "n_chans": {
         "title": "N Chans",
         "type": "integer"
      },
      "n_levels": {
         "title": "N Levels",
         "type": "integer"
      },
      "first_time": {
         "title": "First Time",
         "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v",
         "examples": [
            "2021-01-01 00:00:00.000061_035156_250000_000000"
         ]
      },
      "last_time": {
         "title": "Last Time",
         "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v",
         "examples": [
            "2021-01-01 00:00:00.000061_035156_250000_000000"
         ]
      },
      "system": {
         "title": "System",
         "default": "",
         "type": "string"
      },
      "serial": {
         "title": "Serial",
         "default": "",
         "type": "string"
      },
      "wgs84_latitude": {
         "title": "Wgs84 Latitude",
         "default": -999.0,
         "type": "number"
      },
      "wgs84_longitude": {
         "title": "Wgs84 Longitude",
         "default": -999.0,
         "type": "number"
      },
      "easting": {
         "title": "Easting",
         "default": -999.0,
         "type": "number"
      },
      "northing": {
         "title": "Northing",
         "default": -999.0,
         "type": "number"
      },
      "elevation": {
         "title": "Elevation",
         "default": -999.0,
         "type": "number"
      },
      "chans_metadata": {
         "title": "Chans Metadata",
         "type": "object",
         "additionalProperties": {
            "$ref": "#/definitions/ChanMetadata"
         }
      },
      "levels_metadata": {
         "title": "Levels Metadata",
         "type": "array",
         "items": {
            "$ref": "#/definitions/WindowedLevelMetadata"
         }
      },
      "ref_time": {
         "title": "Ref Time",
         "pattern": "%Y-%m-%d %H:%M:%S.%f_%o_%q_%v",
         "examples": [
            "2021-01-01 00:00:00.000061_035156_250000_000000"
         ]
      },
      "history": {
         "title": "History",
         "default": {
            "records": []
         },
         "allOf": [
            {
               "$ref": "#/definitions/History"
            }
         ]
      }
   },
   "required": [
      "fs",
      "chans",
      "n_levels",
      "first_time",
      "last_time",
      "chans_metadata",
      "levels_metadata",
      "ref_time"
   ],
   "definitions": {
      "ResisticsFile": {
         "title": "ResisticsFile",
         "description": "Required information for writing out a resistics file",
         "type": "object",
         "properties": {
            "created_on_local": {
               "title": "Created On Local",
               "type": "string",
               "format": "date-time"
            },
            "created_on_utc": {
               "title": "Created On Utc",
               "type": "string",
               "format": "date-time"
            },
            "version": {
               "title": "Version",
               "type": "string"
            }
         }
      },
      "ChanMetadata": {
         "title": "ChanMetadata",
         "description": "Channel metadata",
         "type": "object",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            },
            "data_files": {
               "title": "Data Files",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "chan_type": {
               "title": "Chan Type",
               "type": "string"
            },
            "chan_source": {
               "title": "Chan Source",
               "type": "string"
            },
            "sensor": {
               "title": "Sensor",
               "default": "",
               "type": "string"
            },
            "serial": {
               "title": "Serial",
               "default": "",
               "type": "string"
            },
            "gain1": {
               "title": "Gain1",
               "default": 1,
               "type": "number"
            },
            "gain2": {
               "title": "Gain2",
               "default": 1,
               "type": "number"
            },
            "scaling": {
               "title": "Scaling",
               "default": 1,
               "type": "number"
            },
            "chopper": {
               "title": "Chopper",
               "default": false,
               "type": "boolean"
            },
            "dipole_dist": {
               "title": "Dipole Dist",
               "default": 1,
               "type": "number"
            },
            "sensor_calibration_file": {
               "title": "Sensor Calibration File",
               "default": "",
               "type": "string"
            },
            "instrument_calibration_file": {
               "title": "Instrument Calibration File",
               "default": "",
               "type": "string"
            }
         },
         "required": [
            "name"
         ]
      },
      "WindowedLevelMetadata": {
         "title": "WindowedLevelMetadata",
         "description": "Metadata for a windowed level",
         "type": "object",
         "properties": {
            "fs": {
               "title": "Fs",
               "type": "number"
            },
            "n_wins": {
               "title": "N Wins",
               "type": "integer"
            },
            "win_size": {
               "title": "Win Size",
               "exclusiveMinimum": 0,
               "type": "integer"
            },
            "olap_size": {
               "title": "Olap Size",
               "exclusiveMinimum": 0,
               "type": "integer"
            },
            "index_offset": {
               "title": "Index Offset",
               "type": "integer"
            }
         },
         "required": [
            "fs",
            "n_wins",
            "win_size",
            "olap_size",
            "index_offset"
         ]
      },
      "Record": {
         "title": "Record",
         "description": "Class to hold a record\n\nA record holds information about a process that was run. It is intended to\ntrack processes applied to data, allowing a process history to be saved\nalong with any datasets.\n\nExamples\n--------\nA simple example of creating a process record\n\n>>> from resistics.common import Record\n>>> messages = [\"message 1\", \"message 2\"]\n>>> record = Record(\n...     creator={\"name\": \"example\", \"parameter1\": 15},\n...     messages=messages,\n...     record_type=\"example\"\n... )\n>>> record.summary()\n{\n    'time_local': '...',\n    'time_utc': '...',\n    'creator': {'name': 'example', 'parameter1': 15},\n    'messages': ['message 1', 'message 2'],\n    'record_type': 'example'\n}",
         "type": "object",
         "properties": {
            "time_local": {
               "title": "Time Local",
               "type": "string",
               "format": "date-time"
            },
            "time_utc": {
               "title": "Time Utc",
               "type": "string",
               "format": "date-time"
            },
            "creator": {
               "title": "Creator",
               "type": "object"
            },
            "messages": {
               "title": "Messages",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "record_type": {
               "title": "Record Type",
               "type": "string"
            }
         },
         "required": [
            "creator",
            "messages",
            "record_type"
         ]
      },
      "History": {
         "title": "History",
         "description": "Class for storing processing history\n\nParameters\n----------\nrecords : List[Record], optional\n    List of records, by default []\n\nExamples\n--------\n>>> from resistics.testing import record_example1, record_example2\n>>> from resistics.common import History\n>>> record1 = record_example1()\n>>> record2 = record_example2()\n>>> history = History(records=[record1, record2])\n>>> history.summary()\n{\n    'records': [\n        {\n            'time_local': '...',\n            'time_utc': '...',\n            'creator': {\n                'name': 'example1',\n                'a': 5,\n                'b': -7.0\n            },\n            'messages': ['Message 1', 'Message 2'],\n            'record_type': 'process'\n        },\n        {\n            'time_local': '...',\n            'time_utc': '...',\n            'creator': {\n                'name': 'example2',\n                'a': 'parzen',\n                'b': -21\n            },\n            'messages': ['Message 5', 'Message 6'],\n            'record_type': 'process'\n        }\n    ]\n}",
         "type": "object",
         "properties": {
            "records": {
               "title": "Records",
               "default": [],
               "type": "array",
               "items": {
                  "$ref": "#/definitions/Record"
               }
            }
         }
      }
   }
}

field fs: List[float] [Required]
field chans: List[str] [Required]
field n_chans: Optional[int] = None
Validated by

  • validate_n_chans

field n_levels: int [Required]
field first_time: resistics.sampling.HighResDateTime [Required]
Constraints

  • pattern = %Y-%m-%d %H:%M:%S.%f_%o_%q_%v

  • examples = [‘2021-01-01 00:00:00.000061_035156_250000_000000’]

field last_time: resistics.sampling.HighResDateTime [Required]
Constraints

  • pattern = %Y-%m-%d %H:%M:%S.%f_%o_%q_%v

  • examples = [‘2021-01-01 00:00:00.000061_035156_250000_000000’]

field system: str = ''
field serial: str = ''
field wgs84_latitude: float = -999.0
field wgs84_longitude: float = -999.0
field easting: float = -999.0
field northing: float = -999.0
field elevation: float = -999.0
field chans_metadata: Dict[str, resistics.time.ChanMetadata] [Required]
field levels_metadata: List[resistics.window.WindowedLevelMetadata] [Required]
field ref_time: resistics.sampling.HighResDateTime [Required]
Constraints

  • pattern = %Y-%m-%d %H:%M:%S.%f_%o_%q_%v

  • examples = [‘2021-01-01 00:00:00.000061_035156_250000_000000’]

field history: resistics.common.History = History(records=[])
class resistics.window.WindowedData(metadata: resistics.window.WindowedMetadata, data: Dict[int, numpy.ndarray])[source]

Bases: resistics.common.ResisticsData

Windows of a DecimatedData object

The windowed data is stored in a dictionary attribute named data. This is a dictionary with an entry for each decimation level. The shape for a single decimation level is as follows:

n_wins x n_chans x n_samples

get_level(level: int) → numpy.ndarray[source]

Get windows for a decimation level

Parameters

level (int) – The decimation level

Returns

The window array

Return type

np.ndarray

Raises

ValueError – If decimation level is not within range

get_local(level: int, local_win: int) → numpy.ndarray[source]

Get window using local index

Parameters

  • level (int) – The decimation level

  • local_win (int) – Local window index

Returns

Window data

Return type

np.ndarray

Raises

ValueError – If local window index is out of range

get_global(level: int, global_win: int) → numpy.ndarray[source]

Get window using global index

Parameters

  • level (int) – The decimation level

  • global_win (int) – Global window index

Returns

Window data

Return type

np.ndarray

get_chan(level: int, chan: str) → numpy.ndarray[source]

Get all the windows for a channel

Parameters

  • level (int) – The decimation level

  • chan (str) – The channel

Returns

The data for the channels

Return type

np.ndarray

Raises

ChannelNotFoundError – If the channel is not found in the data

to_string() → str[source]

Class information as a string

pydantic model resistics.window.Windower[source]

Bases: resistics.common.ResisticsProcess

Windows DecimatedData

This is the primary window making process for resistics and should be used when alignment of windows with a site or across sites is required.

This method uses numpy striding to produce window views into the decimated data.

See also

WindowerTarget

A windower to make a target number of windows

Examples

The Windower windows a DecimatedData object given a reference time and some window parameters.

There’s quite a few imports needed for this example. Begin by doing the imports, defining a reference time and generating random decimated data.

>>> from resistics.sampling import to_datetime
>>> from resistics.testing import decimated_data_linear
>>> from resistics.window import WindowSetup, Windower
>>> dec_data = decimated_data_linear(fs=128)
>>> ref_time = dec_data.metadata.first_time
>>> print(dec_data.to_string())
<class 'resistics.decimate.DecimatedData'>
           fs        dt  n_samples           first_time                        last_time
level
0      2048.0  0.000488      16384  2021-01-01 00:00:00  2021-01-01 00:00:07.99951171875
1       512.0  0.001953       4096  2021-01-01 00:00:00    2021-01-01 00:00:07.998046875
2       128.0  0.007812       1024  2021-01-01 00:00:00      2021-01-01 00:00:07.9921875

Next, initialise the window parameters. For this example, use small windows, which will make inspecting them easier.

>>> win_params = WindowSetup(win_sizes=[16,16,16], min_olap=4).run(dec_data.metadata.n_levels, dec_data.metadata.fs)
>>> win_params.summary()
{
    'n_levels': 3,
    'min_n_wins': 5,
    'win_sizes': [16, 16, 16],
    'olap_sizes': [4, 4, 4]
}

Perform the windowing. This actually creates views into the decimated data using the numpy.lib.stride_tricks.sliding_window_view function. The shape for a data array at a decimation level is: n_wins x n_chans x win_size. The information about each level is also in the levels_metadata attribute of WindowedMetadata.

>>> win_data = Windower().run(ref_time, win_params, dec_data)
>>> win_data.data[0].shape
(1365, 2, 16)
>>> for level_metadata in win_data.metadata.levels_metadata:
...     level_metadata.summary()
{
    'fs': 2048.0,
    'n_wins': 1365,
    'win_size': 16,
    'olap_size': 4,
    'index_offset': 0
}
{
    'fs': 512.0,
    'n_wins': 341,
    'win_size': 16,
    'olap_size': 4,
    'index_offset': 0
}
{
    'fs': 128.0,
    'n_wins': 85,
    'win_size': 16,
    'olap_size': 4,
    'index_offset': 0
}

Let’s look at an example of data from the first decimation level for the first channel. This is simply a linear set of data ranging from 0…16_383.

>>> dec_data.data[0][0]
array([    0,     1,     2, ..., 16381, 16382, 16383])

Inspecting the first few windows shows they are as expected including the overlap.

>>> win_data.data[0][0, 0]
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15])
>>> win_data.data[0][1, 0]
array([12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27])
>>> win_data.data[0][2, 0]
array([24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39])

Show JSON schema
{
   "title": "Windower",
   "description": "Windows DecimatedData\n\nThis is the primary window making process for resistics and should be used\nwhen alignment of windows with a site or across sites is required.\n\nThis method uses numpy striding to produce window views into the decimated\ndata.\n\nSee Also\n--------\nWindowerTarget : A windower to make a target number of windows\n\nExamples\n--------\nThe Windower windows a DecimatedData object given a reference time and some\nwindow parameters.\n\nThere's quite a few imports needed for this example. Begin by doing the\nimports, defining a reference time and generating random decimated data.\n\n>>> from resistics.sampling import to_datetime\n>>> from resistics.testing import decimated_data_linear\n>>> from resistics.window import WindowSetup, Windower\n>>> dec_data = decimated_data_linear(fs=128)\n>>> ref_time = dec_data.metadata.first_time\n>>> print(dec_data.to_string())\n<class 'resistics.decimate.DecimatedData'>\n           fs        dt  n_samples           first_time                        last_time\nlevel\n0      2048.0  0.000488      16384  2021-01-01 00:00:00  2021-01-01 00:00:07.99951171875\n1       512.0  0.001953       4096  2021-01-01 00:00:00    2021-01-01 00:00:07.998046875\n2       128.0  0.007812       1024  2021-01-01 00:00:00      2021-01-01 00:00:07.9921875\n\nNext, initialise the window parameters. For this example, use small windows,\nwhich will make inspecting them easier.\n\n>>> win_params = WindowSetup(win_sizes=[16,16,16], min_olap=4).run(dec_data.metadata.n_levels, dec_data.metadata.fs)\n>>> win_params.summary()\n{\n    'n_levels': 3,\n    'min_n_wins': 5,\n    'win_sizes': [16, 16, 16],\n    'olap_sizes': [4, 4, 4]\n}\n\nPerform the windowing. This actually creates views into the decimated data\nusing the numpy.lib.stride_tricks.sliding_window_view function. The shape\nfor a data array at a decimation level is: n_wins x n_chans x win_size. The\ninformation about each level is also in the levels_metadata attribute of\nWindowedMetadata.\n\n>>> win_data = Windower().run(ref_time, win_params, dec_data)\n>>> win_data.data[0].shape\n(1365, 2, 16)\n>>> for level_metadata in win_data.metadata.levels_metadata:\n...     level_metadata.summary()\n{\n    'fs': 2048.0,\n    'n_wins': 1365,\n    'win_size': 16,\n    'olap_size': 4,\n    'index_offset': 0\n}\n{\n    'fs': 512.0,\n    'n_wins': 341,\n    'win_size': 16,\n    'olap_size': 4,\n    'index_offset': 0\n}\n{\n    'fs': 128.0,\n    'n_wins': 85,\n    'win_size': 16,\n    'olap_size': 4,\n    'index_offset': 0\n}\n\nLet's look at an example of data from the first decimation level for the\nfirst channel. This is simply a linear set of data ranging from 0...16_383.\n\n>>> dec_data.data[0][0]\narray([    0,     1,     2, ..., 16381, 16382, 16383])\n\nInspecting the first few windows shows they are as expected including the\noverlap.\n\n>>> win_data.data[0][0, 0]\narray([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15])\n>>> win_data.data[0][1, 0]\narray([12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27])\n>>> win_data.data[0][2, 0]\narray([24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39])",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      }
   }
}

run(ref_time: attotime.objects.attodatetime.attodatetime, win_params: resistics.window.WindowParameters, dec_data: resistics.decimate.DecimatedData) → resistics.window.WindowedData[source]

Perform windowing of DecimatedData

Parameters

  • ref_time (RSDateTime) – The reference time

  • win_params (WindowParameters) – The window parameters

  • dec_data (DecimatedData) – The decimated data

Returns

Windows for decimated data

Return type

WindowedData

Raises

ProcessRunError – If the number of windows calculated in the window table does not match the size of the array views

field name: Optional[str] [Required]
Validated by

  • validate_name

pydantic model resistics.window.WindowerTarget[source]

Bases: resistics.window.Windower

Windower that selects window sizes to meet a target number of windows

The minimum window size in window parameters will be respected even if the generated number of windows is below the target. This is to avoid situations where excessively small windows sizes are selected.

Warning

This process is primarily useful for quick processing of a single measurement and should not be used when any alignment of windows is required within a site or across sites.

Parameters

  • target (int) – The target number of windows for each decimation level

  • olap_proportion (float) – The overlap proportion of the window size

See also

Windower

The window making process to use when alignment is required

Show JSON schema
{
   "title": "WindowerTarget",
   "description": "Windower that selects window sizes to meet a target number of windows\n\nThe minimum window size in window parameters will be respected even if the\ngenerated number of windows is below the target. This is to avoid situations\nwhere excessively small windows sizes are selected.\n\n.. warning::\n\n    This process is primarily useful for quick processing of a single\n    measurement and should not be used when any alignment of windows is\n    required within a site or across sites.\n\nParameters\n----------\ntarget : int\n    The target number of windows for each decimation level\nolap_proportion : float\n    The overlap proportion of the window size\n\nSee Also\n--------\nWindower : The window making process to use when alignment is required",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "target": {
         "title": "Target",
         "default": 1000,
         "type": "integer"
      },
      "min_size": {
         "title": "Min Size",
         "default": 64,
         "type": "integer"
      },
      "olap_proportion": {
         "title": "Olap Proportion",
         "default": 0.25,
         "type": "number"
      }
   }
}

field target: int = 1000
field min_size: int = 64
field olap_proportion: float = 0.25
run(ref_time: attotime.objects.attodatetime.attodatetime, win_params: resistics.window.WindowParameters, dec_data: resistics.decimate.DecimatedData) → resistics.window.WindowedData[source]

Perform windowing of DecimatedData

Parameters

  • ref_time (RSDateTime) – The reference time

  • win_params (WindowParameters) – The window parameters

  • dec_data (DecimatedData) – The decimated data

Returns

Windows for decimated data

Return type

WindowedData

Raises

ProcessRunError – If the number of windows calculated in the window table does not match the size of the array views

pydantic model resistics.window.WindowedDataWriter[source]

Bases: resistics.common.ResisticsWriter

Writer of resistics windowed data

Show JSON schema
{
   "title": "WindowedDataWriter",
   "description": "Writer of resistics windowed data",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "overwrite": {
         "title": "Overwrite",
         "default": true,
         "type": "boolean"
      }
   }
}

field overwrite: bool = True
field name: Optional[str] [Required]
Validated by

  • validate_name

run(dir_path: pathlib.Path, win_data: resistics.window.WindowedData) → None[source]

Write out WindowedData

Parameters

  • dir_path (Path) – The directory path to write to

  • win_data (WindowedData) – Windowed data to write out

Raises

WriteError – If unable to write to the directory

pydantic model resistics.window.WindowedDataReader[source]

Bases: resistics.common.ResisticsProcess

Reader of resistics windowed data

Show JSON schema
{
   "title": "WindowedDataReader",
   "description": "Reader of resistics windowed data",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      }
   }
}

field name: Optional[str] [Required]
Validated by

  • validate_name

run(dir_path: pathlib.Path, metadata_only: bool = False) → Union[resistics.window.WindowedMetadata, resistics.window.WindowedData][source]

Read WindowedData

Parameters

  • dir_path (Path) – The directory path to read from

  • metadata_only (bool, optional) – Flag for getting metadata only, by default False

Returns

The WindowedData or WindowedMetadata if metadata_only is True

Return type

Union[WindowedMetadata, WindowedData]

Raises

ReadError – If the directory does not exist