Data Container Pre-process¶

The following provide information on how the primitive data array is built

class crypto_history.data_container.data_container_pre.PrimitiveCoinHistoryObtainer¶

Abstract class to serve as the parent to generating histories obtainer classes

classmethod create_primitive_coin_history_obtainer(exchange_factory: crypto_history.stock_market.stock_market_factory.StockMarketFactory, interval: str, start_time: Union[str, datetime.datetime, int], end_time: Union[str, datetime.datetime, int]) → crypto_history.data_container.data_container_pre.PrimitiveCoinHistoryObtainer¶

Generates the coin history obtainer

Parameters:	exchange_factory (StockMarketFactory) – instance of the exchange_factory which is responsible for setting the market_homogenizer interval (str) – Length of the history of the klines per item start_time (str/datetime/int) – duration from which history is necessary end_time (str/datetime/int) – duration up to which history is necessary
Yields:	PrimitiveCoinHistoryObtainer instance from the above arguments

get_all_raw_tickers() → crypto_history.stock_market.tickers.TickerPool¶

Obtains the ticker pool by accessing the market homogenizer

Returns:	The TickerPool filled with all the ticker available to the market/exchange
Return type:	TickerPool

get_historical_data_from_base_and_reference_assets(base_assets: List[T], reference_assets: List[T]) → Dict[KT, VT]¶

Get the historical data for the combination of base and reference assets :param base_assets: list of all base assets :type base_assets: List[‘str’] :param reference_assets: list of all reference assets :type reference_assets: List[‘str’]

Returns:	The kline historical data of the combinations

initialize_example() → List[T]¶

Initializes an example of the raw history and stores it example_raw_history

Returns:	instance of an example of a raw history of ETHvsBTC
Return type:	list

class crypto_history.data_container.data_container_pre.PrimitiveDimensionsManager¶

Class responsible for managing the dimensions/coordinates of the data container

class Dimensions(base_assets: List[T], reference_assets: List[T], ohlcv_fields: List[T], index_number: List[T])¶: Class for keeping track of the dimensions of the XDataArray

classmethod create_primitive_dimensions_manager(history_obtainer: crypto_history.data_container.data_container_pre.PrimitiveCoinHistoryObtainer) → crypto_history.data_container.data_container_pre.PrimitiveDimensionsManager¶: Initializes the dimensions manager with the coin_history_obtainer

get_depth_of_indices() → List[T]¶

Obtains the depth of the indices to identify how many time-stamp values: each history has available

Returns:	The list of indices from 0..n-1 where n corresponds to the number of time-stamps in the history
Return type:	list

get_dimension_coordinates_from_fields_and_assets(ohlcv_fields: List[T], base_assets: List[T], reference_assets: List[T])¶

Gets the dataclass containing the coordinates required for creating the xr.DataArray :param ohlcv_fields: fields related to open_ts, close_ts, and so on :param base_assets: list of base coins :param reference_assets: list of reference coins

Returns:	DataClass of the coordinates necessary to build the DataArray

static set_coords_dimensions_in_empty_container(dataclass_dimensions_coordinates) → xarray.core.dataarray.DataArray¶: Initialize the xarray container with None values but with the coordinates as it helps in the memory allocation :param dataclass_dimensions_coordinates: dataclass of coordinates/dimensions as the framework for generating the XDataArray

class crypto_history.data_container.data_container_pre.PrimitiveDataArrayOperations¶

Abstract Base Class for generating the data operations

static append_column_to_df(df, column_name, column_value) → pandas.core.frame.DataFrame¶

Appends a new column to a df with constant value :param df: dataframe to which column is to be appended :type df: DataFrame :param column_name: name of the column :type column_name: str :param column_value: value of the column :type column_value: str

Returns:	new column added DataFrame

classmethod create_primitive_data_array_operations(exchange_factory: crypto_history.stock_market.stock_market_factory.StockMarketFactory, base_assets: List[T], reference_assets: List[T], ohlcv_fields: List[T], interval: str, start_time: Union[str, datetime.datetime, int], end_time: Union[str, datetime.datetime, int])¶

Initializes the DataContainerOperations which is the user end-point for the basic data building :param exchange_factory: factory of the exchange :type exchange_factory: StockMarketFactory :param base_assets: list of base coins to be accumulated :type base_assets: List :param reference_assets: list of reference/quote-against coins to be accumulated :type reference_assets: List :param ohlcv_fields: list of fields for the various fields :type ohlcv_fields: List :param interval: data capture interval :type interval: str :param start_time: date from which data collection should start :type start_time: str/datetime/int :param end_time: date up to which data collection should be made :type end_time: str/datetime/int

Yields:	PrimitiveDataArrayOperations instance from the above arguments

get_example_history() → List[T]¶: A standard example on which the gaps are padded

get_populated_primitive_container() → xarray.core.dataarray.DataArray¶

Populates the container and returns it to the use

Returns:	the filled container of information
Return type:	xr.DataArray

populate_container(data_container: xarray.core.dataarray.DataArray, coord_dimension_dataclass) → xarray.core.dataarray.DataArray¶: Populates the xarray.DataArray with the historical data of all the coins

class crypto_history.data_container.utilities.DataFrameOperations¶

Operations purely dedicated to dataframe

static add_extra_rows_to_bottom(df: pandas.core.frame.DataFrame, empty_rows_to_add: int)¶

Adds extra rows to the pd.DataFrame

Parameters:	df (pd.DataFrame) – to which extra rows are to be added empty_rows_to_add (int) – Number of extra rows that have to be added
Returns:	Re-indexed pd.DataFrame which has the compatible number of rows
Return type:	pd.DataFrame

static calculate_rows_to_add(df: pandas.core.frame.DataFrame, list_of_standard_history: List[T]) → int¶

Calculates the additional number of rows that might have to be added to get the df in the same shape

Parameters:	df (pd.DataFrame) – pandas dataframe which is obtained for the coin’s history list_of_standard_history – expected standard history which has the complete history
Returns:	number of rows that have to be added to the df
Return type:	int

static drop_unnecessary_columns_from_df(df, necessary_columns: List[T]) → pandas.core.frame.DataFrame¶

Drop all columns which are not necessary from the df :param df: from which the unnecessary columns are to be dropped :type df: pd.DataFrame :param necessary_columns: list of columns which are to be stored :type necessary_columns: list

Returns:	pd.DataFrame where the unnecessary columns are dropped

get_compatible_df(standard_example: List[T], ticker_history: Iterable[T_co]) → pandas.core.frame.DataFrame¶

Makes the ticker history compatible to the standard pd.DataFrame by extending the shape to add null values

Parameters:	standard_example (DataFrame) – standard example to know how many rows to pad ticker_history – history of the current ticker
Returns:	compatible history of the df adjusted for same rows as expected
Return type:	pd.DataFrame

pad_extra_rows_if_necessary(standard_example: List[T], history_df: pandas.core.frame.DataFrame) → pandas.core.frame.DataFrame¶

Add extra rows on the bottom of the DF is necessary. i.e when the history is incomplete. # FixMe . Probably needs to be reevaluated :param standard_example: standard example on which it is based :type standard_example: DataFrame :param history_df: history of the dataframe that has not been padded yet

Returns:	that has been padded
Return type:	pd.DataFrame