pyemma.coordinates.source¶

pyemma.coordinates.source(inp, features=None, top=None, chunk_size=None)¶

Defines trajectory data source

This function defines input trajectories without loading them. You can pass the resulting object into transformers such as pyemma.coordinates.tica() or clustering algorithms such as pyemma.coordinates.cluster_kmeans(). Then, the data will be streamed instead of being loaded, thus saving memory.

You can also use this function to construct the first stage of a data processing pipeline().

Parameters:	inp (str (file name) or ndarray or list of strings (file names) or list) – of ndarrays The inp file names or input data. Can be given in any of these ways: File name of a single trajectory. It can have any of the molecular dynamics trajectory formats or raw data formats specified in `load()`. List of trajectory file names. It can have any of the molecular dynamics trajectory formats or raw data formats specified in `load()`. Molecular dynamics trajectory in memory as a numpy array of shape (T, N, 3) with T time steps, N atoms each having three (x,y,z) spatial coordinates. List of molecular dynamics trajectories in memory, each given as a numpy array of shape (T_i, N, 3), where trajectory i has T_i time steps and all trajectories have shape (N, 3). Trajectory of some features or order parameters in memory as a numpy array of shape (T, N) with T time steps and N dimensions. List of trajectories of some features or order parameters in memory, each given as a numpy array of shape (T_i, N), where trajectory i has T_i time steps and all trajectories have N dimensions. List of NumPy array files (.npy) of shape (T, N). Note these arrays are not being loaded completely, but mapped into memory (read-only). List of tabulated ASCII files of shape (T, N). features (MDFeaturizer, optional, default = None) – a featurizer object specifying how molecular dynamics files should be read (e.g. intramolecular distances, angles, dihedrals, etc). This parameter only makes sense if the input comes in the form of molecular dynamics trajectories or data, and will otherwise create a warning and have no effect. top (str, optional, default = None) – A topology file name. This is needed when molecular dynamics trajectories are given and no featurizer is given. In this case, only the Cartesian coordinates will be read. chunk_size (int, optional, default = 100 for file readers and 5000 for) – already loaded data The chunk size at which the input file is being processed.
Returns:	reader
Return type:	`ReaderInterface` object

See also

pyemma.coordinates.load(): If your memory is big enough to load all features into memory, don’t bother using source - working in memory is faster!
pyemma.coordinates.pipeline(): The data input is the first stage for your pipeline. Add other stages to it and build a pipeline to analyze big data in streaming mode.

Examples

Create a reader for NumPy files:

>>> import numpy as np
>>> from pyemma.coordinates import source
>>> reader = source(['001.npy', '002.npy'] 

Create a reader for trajectory files and select some distance as feature:

>>> reader = source(['traj01.xtc', 'traj02.xtc'], top='my_structure.pdb') 
>>> reader.featurizer.add_distances([[0, 1], [5, 6]]) 
>>> calculated_features = reader.get_output() 

create a reader for a csv file:

>>> reader = source('data.csv') 

Create a reader for huge NumPy in-memory arrays to process them in huge chunks to avoid memory issues:

>>> data = np.random.random(int(1e7))
>>> reader = source(data, chunk_size=5000)
>>> from pyemma.coordinates import cluster_regspace
>>> regspace = cluster_regspace(reader, dmin=0.1)

class pyemma.coordinates.data.interface.ReaderInterface(chunksize=100)¶

basic interface for readers

Methods

`describe`()	Get a descriptive string representation of this class.
`dimension`()	Returns the number of output dimensions
`fit`(X, **kwargs)	For compatibility with sklearn
`fit_transform`(X, **kwargs)	For compatibility with sklearn
`get_output`([dimensions, stride])	Maps all input data of this transformer and returns it as an array or list of arrays.
`iterator`([stride, lag])	Returns an iterator that allows to access the transformed data.
`n_frames_total`([stride])	Returns the total number of frames, over all trajectories
`number_of_trajectories`()	Returns the number of trajectories
`output_type`()	By default transformers return single precision floats.
`parametrize`([stride])	Parametrize this Transformer
`register_progress_callback`(call_back[, stage])	Registers the progress reporter.
`trajectory_length`(itraj[, stride])	Returns the length of trajectory
`trajectory_lengths`([stride])	Returns the length of each trajectory
`transform`(X)

Attributes

`chunksize`	chunksize defines how much data is being processed at once.
`data_producer`	where the transformer obtains its data.
`in_memory`	are results stored in memory?
`map`
`name`
`ntraj`

chunksize¶: chunksize defines how much data is being processed at once.

data_producer¶: where the transformer obtains its data.

describe()¶: Get a descriptive string representation of this class.

dimension()¶

Returns the number of output dimensions

Returns:

fit(X, **kwargs)¶: For compatibility with sklearn

fit_transform(X, **kwargs)¶: For compatibility with sklearn

get_output(dimensions=slice(0, None, None), stride=1)¶

Maps all input data of this transformer and returns it as an array or list of arrays.

Parameters:	dimensions (list-like of indexes or slice) – indices of dimensions you like to keep, default = all stride (int) – only take every n’th frame, default = 1
Returns:	output – the mapped data, where T is the number of time steps of the input data, or if stride > 1, floor(T_in / stride). d is the output dimension of this transformer. If the input consists of a list of trajectories, Y will also be a corresponding list of trajectories
Return type:	ndarray(T, d) or list of ndarray(T_i, d)

Notes

This function may be RAM intensive if stride is too large or too many dimensions are selected.
if in_memory attribute is True, then results of this methods are cached.

Example

plotting trajectories

>>> import pyemma.coordinates as coor 
>>> import matplotlib.pyplot as plt 

Fill with some actual data!

>>> tica = coor.tica() 
>>> trajs = tica.get_output(dimensions=(0,), stride=100) 
>>> for traj in trajs: 
...     plt.figure() 
...     plt.plot(traj[:, 0]) 

in_memory¶: are results stored in memory?

iterator(stride=1, lag=0)¶

Returns an iterator that allows to access the transformed data.

Parameters:

stride (int) – Only transform every N’th frame, default = 1
lag (int) – Configure the iterator such that it will return time-lagged data with a lag time of lag. If lag is used together with stride the operation will work as if the striding operation is applied before the time-lagged trajectory is shifted by lag steps. Therefore the effective lag time will be stride*lag.

Returns:

iterator – If lag = 0, a call to the .next() method of this iterator will return the pair (itraj, X) : (int, ndarray(n, m)), where itraj corresponds to input sequence number (eg. trajectory index) and X is the transformed data, n = chunksize or n < chunksize at end of input.

If lag > 0, a call to the .next() method of this iterator will return the tuple (itraj, X, Y) : (int, ndarray(n, m), ndarray(p, m)) where itraj and X are the same as above and Y contain the time-lagged data.

Return type:

a TransformerIterator

map = <pyemma.util.annotators.deprecated object>¶

n_frames_total(stride=1)¶

Returns the total number of frames, over all trajectories

Parameters:	stride – return value is the number of frames in trajectories when running through them with a step size of stride
Returns:	the total number of frames, over all trajectories

name¶

ntraj¶

number_of_trajectories()¶

Returns the number of trajectories

Returns:	number of trajectories

output_type()¶: By default transformers return single precision floats.

parametrize(stride=1)¶: Parametrize this Transformer

register_progress_callback(call_back, stage=0)¶

Registers the progress reporter.

Parameters:	call_back (function) – This function will be called with the following arguments: stage (int) instance of pyemma.utils.progressbar.ProgressBar optional args and named keywords (kw), for future changes stage* (int, optional, default=0) – The stage you want the given call back function to be fired.

trajectory_length(itraj, stride=1)¶

Returns the length of trajectory

Parameters:	itraj – trajectory index stride – return value is the number of frames in trajectory when running through it with a step size of stride
Returns:	length of trajectory

trajectory_lengths(stride=1)¶

Returns the length of each trajectory

Parameters:	stride – return value is the number of frames in trajectories when running through them with a step size of stride
Returns:	numpy array containing length of each trajectory

transform(X)¶