diff --git a/docs/databases-and-storage.md b/docs/databases-and-storage.md new file mode 100644 index 0000000..a4356be --- /dev/null +++ b/docs/databases-and-storage.md @@ -0,0 +1,193 @@ +# ARGUS Storage Research + +## Intro + +It was previously a research what ARGUS should store and which database/storage approach fits the project. +Now it's a simple documentation about how the storage and domain logic looks like and should develop in the next sprints. + +ARGUS is moving from live API requests and in-memory analytics toward real data workflows. +The first storage decision should support local market analytics, SQL practice and future dashboard features without adding unnecessary infrastructure too early. + +--- + +## First Storage Approach + +DuckDB should be the first storage technology for ARGUS. + +Reason: + +* ARGUS currently needs local analytical storage, not a full server database +* DuckDB fits historical time-series analysis well +* it supports SQL-based analytics without requiring a database server +* it works well with Python and notebook-based exploration +* it keeps the first storage implementation manageable +* it can later be replaced or complemented by PostgreSQL if ARGUS becomes more product-like + +The first storage implementation should focus on: + +* historical market data +* cleaned OHLCV-ready price data +* source information +* instruments that ARGUS can analyze + +PostgreSQL and SQLGate become more relevant later. + +For the first DuckDB phase, the goal is to build a clean local analytics workflow. + +--- + +## Developer Interaction Workflow + +ARGUS should use a practical developer workflow for DuckDB. + +The goal is to make the database easy to inspect, explore and validate before logic is moved into production code. + +### Notebook Exploration + +Notebooks should be the main exploration layer. + +They are useful for: + +* opening the DuckDB database +* testing SQL queries +* validating imported data +* comparing SQL results with pandas calculations +* exploring metric logic +* documenting research assumptions + +This workflow is especially useful before turning queries into reusable project code. + +Notebook exploration should be preferred over a GUI database tool in the first phase. + +### DuckDB CLI + +The DuckDB CLI should be used for quick database inspection. + +It is useful for: + +* checking available tables +* running small SQL queries +* validating stored records +* debugging the local database file + +The CLI is not the main research environment, but it is useful as a fast inspection tool. + +--- + +## First Data Model Direction + +The first data model should support FX data now and broader market data later. + +ARGUS should not use a narrow `date | value` table as the main market-data model. + +That would work for simple exchange rates, but it would become limiting once ARGUS adds stocks, ETFs, indices or broader market APIs. + +The first model focuses on two primary metadata entities, supported by standard operational schemas and communication interfaces: + +```text +DataSource +Instrument +``` + +### DataSource + +Stores where data came from. + +Current operational fields: + +```text +name +provider_kind +requires_api_key (defaults to False) +``` + +### Instrument + +Stores what ARGUS can analyze. +Current operational fields: + +```text +symbol +name +asset_class +currency (optional) +exchange (optional) +base_currency (optional) +quote_currency (optional) +``` + +### Internal Operational Models + +To support decoupled runtime communication and execution pipelines, ARGUS utilizes specialized Python dataclasses. These structures orchestrate active data flows between data-fetching clients and services. + +**MarketDataRequest** +Encapsulates parameters passed to fetching engines and downstream query handlers. + +* `source`: The targeted `DataSource` instance +* `instrument`: The targeted `Instrument` instance +* `timeframe`: Granularity of requested bars (like `"1d"`, `"1h"`). +* `start`: Temporal lower bound (`datetime.date`) +* `end`: Temporal upper bound (`datetime.date`) + +**MarketDataResponse** +Encapsulates runtime data payloads, structural enforcement, and pipeline feedback. + +* `source`: The originating `DataSource` instance +* `instrument`: The corresponding `Instrument` instance +* `bars`: A `pandas.DataFrame` holding the time-series payload +* `message`: Pipeline feedback or error context (defaults to `""`) + +### The Price Bar Schema (Data Uniformity) + +Crucially, **Price Bars are not treated as a standalone domain metadata model or database entity.** Instead, the Price Bar specification acts purely as an internal **uniformity schema** to normalize incoming time-series data across disparate third-party APIs. + +All responses must be mapped into this standard schema layout within the `bars` DataFrame. + +#### Provider Mapping Layer + +Third-party structures are converted to this unified standard upon ingest. For example, `yfinance` fields map to the uniform schema via `YFINANCE_PRICE_BAR_MAPPING`: + +* `Date` $\rightarrow$ `timestamp` +* `Open` $\rightarrow$ `open` +* `High` $\rightarrow$ `high` +* `Low` $\rightarrow$ `low` +* `Close` $\rightarrow$ `close` +* `Adj Close` $\rightarrow$ `adjusted_close` +* `Volume` $\rightarrow$ `volume` + +--- + +## Future Direction + +Later sprints can expand the storage layer step by step. + +Possible later additions: + +| Future Area | Possible Additions | +| --- | --- | +| Better source mapping | source-specific symbols, provider metadata | +| Watchlists | user-selected instruments | +| Reports | generated report metadata and history | +| Macro data | FRED indicators and observations | +| Paper trading | simulated orders, positions and portfolio history | +| Server architecture | PostgreSQL | +| SQL tooling | SQLGate with PostgreSQL | +| Cloud direction | managed PostgreSQL or cloud storage | + +SQLGate should be kept for a later PostgreSQL phase. + +It becomes useful when ARGUS moves toward: + +* server-based storage +* stronger database management +* richer metadata +* more stable application state +* user-facing features +* report history +* cloud-ready architecture + +Additional metadata such as documentation links, terms links or provider governance fields can also become useful later. + +For the first DuckDB phase, these details should stay in research documentation instead of the database schema. + +--- diff --git a/docs/research-databases-and-storage.md b/docs/research-databases-and-storage.md deleted file mode 100644 index 484cd61..0000000 --- a/docs/research-databases-and-storage.md +++ /dev/null @@ -1,388 +0,0 @@ -# ARGUS Storage Research - -## Goal - -Research what ARGUS should store and which database/storage approach fits the project. - -ARGUS is moving from live API requests and in-memory analytics toward real data workflows. -The first storage decision should support local market analytics, SQL practice and future dashboard features without adding unnecessary infrastructure too early. - ---- - -## Storage Use Cases - -ARGUS should eventually store different kinds of data, but not all of them need to be implemented at once. - -Relevant storage use cases are: - -* historical exchange rates -* cleaned historical market data -* source information -* instruments that ARGUS can analyze -* later watchlists -* later generated reports -* later macroeconomic data -* later paper-trading history - -The first implementation should focus on historical market data and the basic entities needed to query it. - ---- - -## Storage Candidates - -ARGUS should compare storage options based on the current project phase. - -The project currently needs local analytical storage, not a full server or cloud database. - -### DuckDB - -DuckDB is a local analytical database. - -It is a strong fit for ARGUS because it supports SQL-based analytics without requiring a database server. - -Useful for: - -* historical market data -* local time-series analysis -* SQL practice -* Python-based analytics -* notebook-based exploration -* dashboard data preparation - -Limitations: - -* not a server database -* less suitable for multi-user product features later - ---- - -### SQLite - -SQLite is a simple local database. - -It is strong for small app storage and simple persistence. - -Useful for: - -* settings -* small app-state data -* simple local tables -* later watchlists -* lightweight metadata - -Limitations: - -* less analytics-focused than DuckDB -* not ideal as the main storage layer for historical market data -* better for app-state than analytical time-series queries - ---- - -### PostgreSQL - -PostgreSQL is a server-based relational database. - -It is a strong long-term option when ARGUS becomes more product-like. - -Useful for: - -* server-based storage -* user-facing features -* report history -* watchlists -* paper-trading history -* richer metadata -* cloud-ready architecture -* SQLGate usage later - -Limitations: - -* more setup than needed right now -* requires server or Docker setup -* adds infrastructure complexity too early - -Fit for ARGUS: - -PostgreSQL should be introduced later when ARGUS moves toward a server-based or cloud-ready architecture. - ---- - -## Local, Server and Cloud Options - -| Option | Meaning | Fit Now | Fit Later | -| --- | --- | ---: | ---: | -| Local storage | Database runs locally inside or next to the project | High | High | -| Server database | Database runs as a separate service, for example PostgreSQL | Medium | High | -| Cloud storage/database | Managed storage or database in the cloud | Low | High | - -ARGUS should start with local storage. - -Reason: - -* simpler setup -* easier learning curve -* good fit for a Python analytics project -* no cloud or server infrastructure required yet -* enough for historical data, metrics and dashboard development - -Server and cloud storage should come later when ARGUS has stronger product features such as reports, user state, paper-trading history or deployment needs. - ---- - -## Recommended First Storage Approach - -DuckDB should be the first storage technology for ARGUS. - -Reason: - -* ARGUS currently needs local analytical storage, not a full server database -* DuckDB fits historical time-series analysis well -* it supports SQL-based analytics without requiring a database server -* it works well with Python and notebook-based exploration -* it keeps the first storage implementation manageable -* it can later be replaced or complemented by PostgreSQL if ARGUS becomes more product-like - -The first storage implementation should focus on: - -* historical market data -* cleaned OHLCV-ready price data -* source information -* instruments that ARGUS can analyze - -PostgreSQL and SQLGate become more relevant later. - -For the first DuckDB phase, the goal is to build a clean local analytics workflow. - ---- - -## Developer Interaction Workflow - -ARGUS should use a practical developer workflow for DuckDB. - -The goal is to make the database easy to inspect, explore and validate before logic is moved into production code. - -### Notebook Exploration - -Notebooks should be the main exploration layer. - -They are useful for: - -* opening the DuckDB database -* testing SQL queries -* validating imported data -* comparing SQL results with pandas calculations -* exploring metric logic -* documenting research assumptions - -This workflow is especially useful before turning queries into reusable project code. - -Notebook exploration should be preferred over a GUI database tool in the first phase. - -### DuckDB CLI - -The DuckDB CLI should be used for quick database inspection. - -It is useful for: - -* checking available tables -* running small SQL queries -* validating stored records -* debugging the local database file - -The CLI is not the main research environment, but it is useful as a fast inspection tool. - -A GUI tool such as DBeaver can be tested if needed, but it should stay optional. - ---- - -## First Data Model Direction - -The first data model should support FX data now and broader market data later. - -ARGUS should not use a narrow `date | value` table as the main market-data model. - -That would work for simple exchange rates, but it would become limiting once ARGUS adds stocks, ETFs, indices or broader market APIs. - -The first model should focus on three related entities: - -```text -data_sources -instruments -price_bars -``` - -> [!NOTE] -> The fields below describe the future database-oriented structure. -> Technical fields such as `id`, `instrument_id`, `source_id`, `created_at` and `updated_at` are expected to appear in the database layer. -> Internal Python models may reference related objects directly, for example `source` and `instrument`, before database IDs exist. - -### data_sources - -Stores where data came from. - -Recommended first database fields: - -```text -id -name -provider_kind -requires_api_key -created_at -updated_at -``` - -Example internal/source records: - -| name | provider_kind | requires_api_key | -| ---------------- | ------------- | ---------------: | -| ExchangeRate API | fx_rates | true | -| yfinance | market_prices | false | -| FRED | macro_data | true | - -### instruments - -Stores what ARGUS can analyze. - -Examples: - -* EUR/USD -* AAPL -* SPY -* S&P 500 -* BTC-USD - -Recommended first database fields: - -```text -id -symbol -name -asset_class -currency -exchange -base_currency -quote_currency -created_at -updated_at -``` - -Example instrument records: - -| symbol | name | asset_class | currency | exchange | base_currency | quote_currency | -| ------- | ---------------- | ----------- | -------- | --------- | ------------- | -------------- | -| EUR/USD | Euro / US Dollar | fx | null | null | EUR | USD | -| AAPL | Apple Inc. | stock | USD | NASDAQ | null | null | -| SPY | SPDR S&P 500 ETF | etf | USD | NYSE Arca | null | null | - -### price_bars - -Stores historical market data in an OHLCV-ready structure. - -Recommended first database fields: - -```text -id -instrument_id -source_id -timestamp -timeframe -open -high -low -close -adjusted_close -volume -created_at -updated_at -``` - -FX-style exchange-rate data can be represented as a price bar by storing the rate in `close`. - -The other OHLCV fields can stay empty until ARGUS uses data sources that provide them. - -Example price bar records shown with joined source and instrument information for readability: - -| source | instrument | timestamp | timeframe | open | high | low | close | adjusted_close | volume | -| -------- | ---------- | ---------- | --------- | -----: | -----: | -----: | -----: | -------------: | -------: | -| yfinance | EUR/USD | 2024-01-02 | 1d | null | null | null | 1.095 | null | null | -| yfinance | AAPL | 2024-01-02 | 1d | 187.15 | 188.44 | 183.89 | 185.64 | 184.25 | 50200000 | - ---- - -## Recommended First Implementation Step - -The first storage implementation should not be tied to one specific data provider. - -ARGUS currently works with an existing ExchangeRate API client and evaluates broader market data through yfinance. -Frankfurter may be added later as a stronger FX-oriented historical data source. - -The storage layer should therefore focus on a normalized internal market-data format instead of depending on one API response structure. - -Recommended first step: - -```text -active data client -→ normalize into instruments and price_bars -→ store in DuckDB -→ query with SQL -→ use results for analytics and charts -``` - ---- - -## Future Direction - -Later sprints can expand the storage layer step by step. - -Possible later additions: - -| Future Area | Possible Additions | -| --- | --- | -| Better source mapping | source-specific symbols, provider metadata | -| Watchlists | user-selected instruments | -| Reports | generated report metadata and history | -| Macro data | FRED indicators and observations | -| Paper trading | simulated orders, positions and portfolio history | -| Server architecture | PostgreSQL | -| SQL tooling | SQLGate with PostgreSQL | -| Cloud direction | managed PostgreSQL or cloud storage | - -SQLGate should be kept for a later PostgreSQL phase. - -It becomes useful when ARGUS moves toward: - -* server-based storage -* stronger database management -* richer metadata -* more stable application state -* user-facing features -* report history -* cloud-ready architecture - -Additional metadata such as documentation links, terms links or provider governance fields can also become useful later. - -For the first DuckDB phase, these details should stay in research documentation instead of the database schema. - ---- - -## Final Recommendation - -ARGUS should start with DuckDB as the first local analytics storage layer. - -DuckDB fits the current phase best because ARGUS needs local analytical SQL workflows, not a full server database yet. - -The first implementation should store historical market data in an OHLCV-ready structure. - -The recommended first data model is: - -```text -data_sources -instruments -price_bars -``` - -Notebook exploration should be the main developer workflow before SQL logic is moved into application code. - -The DuckDB CLI can be used for quick inspection. - -PostgreSQL and SQLGate should be introduced later when ARGUS moves toward a more product-like or cloud-based architecture. diff --git a/src/argus/analytics/charts/trend_chart.py b/src/argus/analytics/charts/trend_chart.py index 1293361..db5dacb 100644 --- a/src/argus/analytics/charts/trend_chart.py +++ b/src/argus/analytics/charts/trend_chart.py @@ -1,8 +1,8 @@ import matplotlib.pyplot as plt -from argus.services.timeseries_service import prepare_trend_analysis +import pandas as pd -def create_trendchart(curr_symbol: str, start: str, end: str, interval: str): +def create_trendchart(df: pd.DataFrame, min_max_rates: dict): """ Create a trend chart for exchange-rate analysis. @@ -30,10 +30,6 @@ def create_trendchart(curr_symbol: str, start: str, end: str, interval: str): Minimum and maximum exchange-rate values are marked with scatter points and annotations. """ - result = prepare_trend_analysis(curr_symbol, start, end, interval) - if result is None: - return None - df, min_max_rates = result min_date = min_max_rates["min_date"][0] min_rate = min_max_rates["min_rate"][0] max_date = min_max_rates["max_date"][0] diff --git a/src/argus/clients/yfinance_client.py b/src/argus/clients/yfinance_client.py index de86041..468b87b 100644 --- a/src/argus/clients/yfinance_client.py +++ b/src/argus/clients/yfinance_client.py @@ -1,43 +1,48 @@ import yfinance as yf -import logging - - -def get_timeseries(curr_symbol, start, end, interval): - """ - Fetch historical exchange-rate time series data from Yahoo Finance. - - Args: - curr_symbol (str): Currency symbol used by Yahoo Finance, for example - "EURUSD=X". - start (str): Start date of the requested time range in YYYY-MM-DD format. - end (str): End date of the requested time range in YYYY-MM-DD format. - interval (str): Data interval supported by Yahoo Finance, for example - "1d", "1h", or "15m". - - Returns: - pandas.DataFrame | None: A DataFrame containing the columns ``date`` and - ``rate`` if data was successfully fetched. Returns ``None`` if the - request fails, returns no data, or an exception occurs. - """ +import pandas as pd +from argus.domain.internal_models import ( + MarketDataRequest, + PRICE_BAR_COLUMNS, + YFINANCE_PRICE_BAR_MAPPING, +) + + +def get_timeseries(request: MarketDataRequest) -> pd.DataFrame: + start = str(request.start) + end = str(request.end) + timeframe = request.timeframe + curr_pair = ( + f"{request.instrument.base_currency}{request.instrument.quote_currency}=X" + ) + try: - yf_logger = logging.getLogger("yfinance") - yf_logger.disabled = True - data = yf.download( - tickers=curr_symbol, + raw_resp = yf.download( + tickers=curr_pair, start=start, end=end, - interval=interval, + interval=timeframe, multi_level_index=False, progress=False, ) - yf_logger.disabled = False - if data is None: - return None - if data.empty: - return None - data = data.reset_index() - data = data[["Date", "Close"]] - data = data.rename(columns={"Date": "date", "Close": "rate"}) - return data except Exception: - return None + raise ConnectionError("Network error or connection timeout") + + if raw_resp is None: + raise ConnectionError("Yahoo Finance API returned an invalid response") + + if ( + raw_resp.empty + or "Close" not in raw_resp.columns + or raw_resp["Close"].dropna().empty + ): + raise ValueError("Quote not found or no data available for symbol") + + resp = normalize_yfinance_bars(raw_resp) + return resp + + +def normalize_yfinance_bars(raw_df: pd.DataFrame) -> pd.DataFrame: + df = raw_df.copy() + df = df.reset_index() + df = df.rename(columns=YFINANCE_PRICE_BAR_MAPPING) + return df[list(PRICE_BAR_COLUMNS)] diff --git a/src/argus/domain/internal_models.py b/src/argus/domain/internal_models.py index 3b7630e..0d64796 100644 --- a/src/argus/domain/internal_models.py +++ b/src/argus/domain/internal_models.py @@ -1,5 +1,6 @@ from dataclasses import dataclass from datetime import date +import pandas as pd @dataclass @@ -20,15 +21,51 @@ class Instrument: quote_currency: str | None = None +PRICE_BAR_COLUMNS = ( + "timestamp", + "open", + "high", + "low", + "close", + "adjusted_close", + "volume", +) + +YFINANCE_PRICE_BAR_MAPPING = { + "Date": "timestamp", + "Open": "open", + "High": "high", + "Low": "low", + "Close": "close", + "Adj Close": "adjusted_close", + "Volume": "volume", +} + + @dataclass -class PriceBar: +class MarketDataRequest: source: DataSource instrument: Instrument - timestamp: date timeframe: str - close: float - open: float | None = None - high: float | None = None - low: float | None = None - adjusted_close: float | None = None - volume: float | None = None + start: date + end: date + + +@dataclass +class MarketDataResponse: + source: DataSource + instrument: Instrument + bars: pd.DataFrame + message: str = "" + + def __post_init__(self) -> None: + if not isinstance(self.bars, pd.DataFrame): + raise TypeError("bars must be a pandas DataFrame") + if self.message == "": + missing_cols = [ + col for col in PRICE_BAR_COLUMNS if col not in self.bars.columns + ] + if missing_cols: + raise ValueError( + f"Missing required columns in bars DataFrame: {missing_cols}" + ) diff --git a/src/argus/gui/app.py b/src/argus/gui/app.py index 9a19523..02f4c1f 100644 --- a/src/argus/gui/app.py +++ b/src/argus/gui/app.py @@ -1,6 +1,7 @@ import tkinter as tk +import pandas as pd from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg -from argus.analytics.charts.trend_chart import create_trendchart +from argus.services.market_data_service import prepare_trend_analysis from argus.services.calculator_service import calc, check_op from argus.services.conversion_service import convert, check_currency from argus.domain.validation import parse_amount @@ -76,11 +77,6 @@ def show_trend() -> None: global trend_canvas global trend_chart_widget - curr_symbol = "EURUSD=X" - start = "2024-01-01" - end = "2025-01-01" - interval = "1d" - calc_frame.pack_forget() conv_frame.pack_forget() menu_frame.pack_forget() @@ -90,7 +86,8 @@ def show_trend() -> None: content.pack(side="top", fill=tk.BOTH, expand=True) if trend_canvas is None: - fig = create_trendchart(curr_symbol, start, end, interval) + df = pd.DataFrame() + fig = prepare_trend_analysis(df) if fig is None: return None fig.set_size_inches(7, 4) diff --git a/src/argus/main.py b/src/argus/main.py index 7130dbf..105de61 100644 --- a/src/argus/main.py +++ b/src/argus/main.py @@ -1,11 +1,14 @@ from argus.gui.app import app +from argus.storage.database import initialize_database -def main() -> None: +def main(db) -> None: """ The main function that starts the application. """ + initialize_database(db) app() -main() +db = "" +main(db) diff --git a/src/argus/services/market_data_service.py b/src/argus/services/market_data_service.py new file mode 100644 index 0000000..9328141 --- /dev/null +++ b/src/argus/services/market_data_service.py @@ -0,0 +1,47 @@ +from argus.domain.internal_models import MarketDataRequest, MarketDataResponse +from argus.clients.yfinance_client import get_timeseries +from argus.storage.database import read_price_bars, insert_price_bar +import pandas as pd + + +def get_market_data( + db: str, + request: MarketDataRequest, +) -> MarketDataResponse: + """ + Get a time series either from local stroage or client with first-storage-workflow + + Args: + curr_symbol (str): Currency symbol used by Yahoo Finance, for example + "EURUSD=X". + start (str): Start date of the requested time range in YYYY-MM-DD format. + end (str): End date of the requested time range in YYYY-MM-DD format. + intervall (str): Data interval supported by Yahoo Finance, for example + "1d", "1h", or "15m". + + Returns: + pd.DataFrame | None: A + DataFrame with dates and rates. Returns + ``None`` if no time-series data could be fetched. + """ + bars = read_price_bars(db, request) + if not (bars.empty): + db_response = MarketDataResponse( + source=request.source, instrument=request.instrument, bars=bars, message="" + ) + return db_response + + try: + bars = get_timeseries(request) + api_response = MarketDataResponse( + source=request.source, instrument=request.instrument, bars=bars + ) + insert_price_bar(db, api_response) + return api_response + except (ConnectionError, ValueError) as e: + return MarketDataResponse( + source=request.source, + instrument=request.instrument, + bars=pd.DataFrame(), + message=str(e), + ) diff --git a/src/argus/services/timeseries_service.py b/src/argus/services/timeseries_service.py deleted file mode 100644 index b6251bb..0000000 --- a/src/argus/services/timeseries_service.py +++ /dev/null @@ -1,41 +0,0 @@ -import pandas as pd -from argus.clients.yfinance_client import get_timeseries -from argus.analytics.metrics.trend_metrics import ( - add_rolling_average, - add_daily_percentage_change, - get_min_max_rates, -) - - -def prepare_trend_analysis( - curr_symbol: str, start: str, end: str, intervall: str -) -> tuple[pd.DataFrame, dict] | None: - """ - Prepare time-series data for trend analysis. - - Fetches historical exchange-rate data for the given currency symbol and - enriches it with daily percentage changes and a rolling average. It also - calculates the minimum and maximum exchange rates for the resulting time - series. - - Args: - curr_symbol (str): Currency symbol used by Yahoo Finance, for example - "EURUSD=X". - start (str): Start date of the requested time range in YYYY-MM-DD format. - end (str): End date of the requested time range in YYYY-MM-DD format. - intervall (str): Data interval supported by Yahoo Finance, for example - "1d", "1h", or "15m". - - Returns: - tuple[pd.DataFrame, dict] | None: A tuple containing the prepared - DataFrame and a dictionary with minimum and maximum rates. Returns - ``None`` if no time-series data could be fetched. - """ - - df = get_timeseries(curr_symbol, start, end, intervall) - if df is None: - return None - df = add_daily_percentage_change(df) - df = add_rolling_average(df) - min_max_rates = get_min_max_rates(df) - return df, min_max_rates diff --git a/src/argus/services/trend_analysis_service.py b/src/argus/services/trend_analysis_service.py new file mode 100644 index 0000000..a8428ae --- /dev/null +++ b/src/argus/services/trend_analysis_service.py @@ -0,0 +1,37 @@ +from argus.analytics.metrics.trend_metrics import ( + add_rolling_average, + add_daily_percentage_change, + get_min_max_rates, +) +from matplotlib.figure import Figure +from argus.analytics.charts.trend_chart import create_trendchart +from argus.services.market_data_service import get_market_data +from argus.domain.internal_models import MarketDataRequest + + +def prepare_trend_analysis(db: str, request: MarketDataRequest) -> Figure | str: + """ + Prepare time-series data and generate a trend analysis chart. + + Enriches the historical exchange-rate DataFrame with daily percentage changes + and a rolling average, calculates the minimum and maximum rates, and uses + the result to build a trend visualization chart. + + Args: + df (pd.DataFrame): A DataFrame containing market data time-series. + + Returns: + plotly.graph_objects.Figure: A figure object representing the + generated trend chart. + """ + response = get_market_data(db, request) + + if response.message: + return response.message + + df = response.bars.copy() + df = add_daily_percentage_change(df) + df = add_rolling_average(df) + min_max_rates = get_min_max_rates(df) + fig = create_trendchart(df, min_max_rates) + return fig diff --git a/src/argus/storage/database.py b/src/argus/storage/database.py index ea7128c..aa00c78 100644 --- a/src/argus/storage/database.py +++ b/src/argus/storage/database.py @@ -1,7 +1,11 @@ import duckdb -from datetime import date import pandas as pd -from argus.domain.internal_models import DataSource, PriceBar, Instrument +from argus.domain.internal_models import ( + DataSource, + Instrument, + MarketDataRequest, + MarketDataResponse, +) def initialize_database(database_path: str) -> None: @@ -47,7 +51,6 @@ def initialize_database(database_path: str) -> None: source_id INTEGER NOT NULL, instrument_id INTEGER NOT NULL, timestamp DATE NOT NULL, - timeframe TEXT NOT NULL, close DOUBLE NOT NULL, open DOUBLE, high DOUBLE, @@ -56,7 +59,7 @@ def initialize_database(database_path: str) -> None: volume DOUBLE, FOREIGN KEY (source_id) REFERENCES data_sources (id), FOREIGN KEY (instrument_id) REFERENCES instruments (id), - UNIQUE (source_id, instrument_id, timestamp, timeframe) + UNIQUE (source_id, instrument_id, timestamp,close) ); """, ] @@ -186,13 +189,27 @@ def get_or_create_instrument(connection, instrument: Instrument) -> int: return result[0] -def insert_price_bar(db: str, price_bar: PriceBar) -> None: +def insert_price_bar(db: str, marketdata: MarketDataResponse) -> None: + """ + Insert a price bar into the database. + + Ensures that the related data source and instrument exist, then inserts + the price bar into the ``price_bars`` table. Duplicate price bars are + ignored through the table's unique constraint. + + Args: + db (str): Path to the DuckDB database file. + price_bar (PriceBar): Price bar model containing source, + instrument, timestamp and market values. + + Returns: + None + """ insert_query = """ INSERT INTO price_bars ( source_id, instrument_id, timestamp, - timeframe, close, open, high, @@ -200,46 +217,58 @@ def insert_price_bar(db: str, price_bar: PriceBar) -> None: adjusted_close, volume ) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT DO NOTHING; """ connection = duckdb.connect(db) try: - source_id = get_or_create_source(connection, price_bar.source) - instrument_id = get_or_create_instrument(connection, price_bar.instrument) - connection.execute( - query=insert_query, - parameters=[ - source_id, - instrument_id, - price_bar.timestamp, - price_bar.timeframe, - price_bar.close, - price_bar.open, - price_bar.high, - price_bar.low, - price_bar.adjusted_close, - price_bar.volume, - ], - ) + source_id = get_or_create_source(connection, marketdata.source) + instrument_id = get_or_create_instrument(connection, marketdata.instrument) + if marketdata.bars is None: + return None + for _, row in marketdata.bars.iterrows(): + connection.execute( + query=insert_query, + parameters=[ + source_id, + instrument_id, + row["timestamp"], + row["close"], + row["open"], + row["high"], + row["low"], + row["adjusted_close"], + row["volume"], + ], + ) finally: connection.close() -def read_price_bars( - db: str, - source: DataSource, - instrument: Instrument, - start_date: date, - end_date: date, -) -> pd.DataFrame: +def read_price_bars(db: str, request: MarketDataRequest) -> pd.DataFrame: + """ + Read price bars for a source, instrument, and date range. + + Queries stored price bars joined with their data source and instrument + metadata. Results are ordered by timestamp and returned as a pandas + DataFrame. + + Args: + db (str): Path to the DuckDB database file. + source (DataSource): Data source used to filter the stored price bars. + instrument (Instrument): Instrument used to filter the stored price bars. + start_date (date): Inclusive start date of the requested time range. + end_date (date): Inclusive end date of the requested time range. + + Returns: + pd.DataFrame: DataFrame containing matching price bars and metadata. + """ search_query = """ SELECT data_sources.name AS source_name, instruments.symbol AS instrument_symbol, price_bars.timestamp, - price_bars.timeframe, price_bars.open, price_bars.high, price_bars.low, @@ -260,10 +289,10 @@ def read_price_bars( result = connection.execute( query=search_query, parameters=[ - source.name, - instrument.symbol, - start_date, - end_date, + request.source.name, + request.instrument.symbol, + request.start, + request.end, ], ).df() finally: diff --git a/tests/test_internal_models.py b/tests/test_internal_models.py index 97df4c6..0fc7f0c 100644 --- a/tests/test_internal_models.py +++ b/tests/test_internal_models.py @@ -1,101 +1,66 @@ -from argus.domain.internal_models import DataSource, Instrument, PriceBar +import pytest +import pandas as pd from datetime import date +from argus.domain.internal_models import DataSource, Instrument, MarketDataResponse -def test_data_source_can_be_created() -> None: - source = DataSource( - name="yfinance", - provider_kind="fx_rates", - ) - - assert source.name == "yfinance" - assert source.provider_kind == "fx_rates" - assert source.requires_api_key is False +@pytest.fixture +def valid_source(): + return DataSource(name="Yahoo", provider_kind="yfinance_api") -def test_instrument_can_be_created() -> None: - instrument = Instrument( - symbol="EUR/USD", - name="Euro / US Dollar", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) +@pytest.fixture +def valid_instrument(): + return Instrument(symbol="AAPL", name="Apple Inc.", asset_class="stock") - assert instrument.symbol == "EUR/USD" - assert instrument.name == "Euro / US Dollar" - assert instrument.asset_class == "fx" - assert instrument.base_currency == "EUR" - assert instrument.quote_currency == "USD" - assert instrument.currency is None - assert instrument.exchange is None +def test_market_data_response_accepts_valid_dataframe( + valid_source, valid_instrument +) -> None: + valid_bar = { + "timestamp": [date(2026, 1, 1)], + "open": [150.0], + "high": [155.0], + "low": [149.0], + "close": [153.5], + "adjusted_close": [153.5], + "volume": [1000000.0], + } + df = pd.DataFrame(valid_bar) -def test_rate_bar_can_be_created() -> None: - source = DataSource( - name="yfinance", - provider_kind="fx_rates", + resp = MarketDataResponse( + source=valid_source, instrument=valid_instrument, bars=df, message="" ) + assert resp.bars.equals(df) - instrument_rate = Instrument( - symbol="EUR/USD", - name="Euro / US Dollar", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) - price_bar = PriceBar( - source=source, - instrument=instrument_rate, - timestamp=date(2026, 1, 1), - timeframe="1d", - close=1.89, - ) +def test_market_data_response_raises_error_on_missing_columns( + valid_source, valid_instrument +) -> None: + incomplete_bar = { + "timestamp": [date(2026, 1, 1)], + "close": [153.5], + } + df = pd.DataFrame(incomplete_bar) - assert price_bar.source == source - assert price_bar.instrument == instrument_rate - assert price_bar.timestamp == date(2026, 1, 1) - assert price_bar.timeframe == "1d" - assert price_bar.close == 1.89 - assert price_bar.open is None - assert price_bar.high is None - assert price_bar.low is None - assert price_bar.adjusted_close is None - assert price_bar.volume is None + with pytest.raises(ValueError) as exc_info: + MarketDataResponse( + source=valid_source, instrument=valid_instrument, bars=df, message="" + ) + assert "Missing required columns" in str(exc_info.value) -def test_stock_ohlcv_bar_can_be_created() -> None: - source = DataSource( - name="yfinance", - provider_kind="market_prices", - ) - instrument = Instrument( - symbol="AAPL", - name="Apple Inc.", - asset_class="stock", - currency="USD", - exchange="NASDAQ", - ) +def test_market_data_response_raises_error_if_not_a_dataframe( + valid_source, valid_instrument +) -> None: + invalid_input = "I'm just a string :D" - price_bar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), - timeframe="1d", - open=187.15, - high=188.44, - low=183.89, - close=185.64, - adjusted_close=184.25, - volume=50_200_000, - ) + with pytest.raises(TypeError) as exc_info: + MarketDataResponse( + source=valid_source, + instrument=valid_instrument, + bars=invalid_input, # type: ignore + ) - assert price_bar.instrument.symbol == "AAPL" - assert price_bar.open == 187.15 - assert price_bar.high == 188.44 - assert price_bar.low == 183.89 - assert price_bar.close == 185.64 - assert price_bar.adjusted_close == 184.25 - assert price_bar.volume == 50_200_000 + assert "must be a pandas DataFrame" in str(exc_info.value) diff --git a/tests/test_market_data_series.py b/tests/test_market_data_series.py new file mode 100644 index 0000000..b7832a2 --- /dev/null +++ b/tests/test_market_data_series.py @@ -0,0 +1,138 @@ +import pytest +import pandas as pd +from datetime import date +from unittest.mock import Mock +from argus.domain.internal_models import ( + DataSource, + Instrument, + MarketDataRequest, + MarketDataResponse, +) +from argus.services.market_data_service import get_market_data + + +@pytest.fixture +def sample_source(): + return DataSource(name="Yahoo", provider_kind="yfinance_api") + + +@pytest.fixture +def sample_instrument(): + return Instrument(symbol="AAPL", name="Apple Inc.", asset_class="stock") + + +@pytest.fixture +def sample_response(sample_source, sample_instrument): + test_bar = { + "timestamp": date(2026, 1, 1), + "open": None, + "high": None, + "low": None, + "close": 1.89, + "adjusted_close": None, + "volume": None, + } + return MarketDataResponse( + source=sample_source, + instrument=sample_instrument, + bars=pd.DataFrame(test_bar, index=[0]), + ) + + +def test_get_market_data_storage_hit( + monkeypatch, sample_source, sample_instrument, sample_response +): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2026, 1, 1), + end=date(2026, 1, 1), + ) + + monkeypatch.setattr( + "argus.services.market_data_service.read_price_bars", + lambda db, r: sample_response.bars, + ) + + mock_get_timeseries = Mock() + monkeypatch.setattr( + "argus.services.market_data_service.get_timeseries", mock_get_timeseries + ) + + res = get_market_data("mock_db_path", req) + + assert res is not None + assert not res.bars.empty + mock_get_timeseries.assert_not_called() + + +def test_get_market_data_storage_miss( + monkeypatch, sample_source, sample_instrument, sample_response +): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2026, 1, 1), + end=date(2026, 1, 1), + ) + + monkeypatch.setattr( + "argus.services.market_data_service.read_price_bars", + lambda db, r: pd.DataFrame(), + ) + monkeypatch.setattr( + "argus.services.market_data_service.get_timeseries", + lambda r: sample_response.bars, + ) + + mock_insert = Mock() + monkeypatch.setattr( + "argus.services.market_data_service.insert_price_bar", mock_insert + ) + + res = get_market_data("mock_db_path", req) + + assert res is not None + mock_insert.assert_called_once() + + +def test_get_market_data_handles_client_exceptions( + monkeypatch, sample_source, sample_instrument +): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2026, 1, 1), + end=date(2026, 1, 1), + ) + + monkeypatch.setattr( + "argus.services.market_data_service.read_price_bars", + lambda db, r: pd.DataFrame(), + ) + + def mock_value_error(request): + raise ValueError("Quote not found or no data available for symbol") + + monkeypatch.setattr( + "argus.services.market_data_service.get_timeseries", mock_value_error + ) + + res_val = get_market_data("mock_db_path", req) + assert res_val.bars.empty + assert res_val.message == "Quote not found or no data available for symbol" + + # Fall 2: ConnectionError testen (z.B. Internet weg) + def mock_connection_error(request): + raise ConnectionError("Network error or connection timeout") + + monkeypatch.setattr( + "argus.services.market_data_service.get_timeseries", mock_connection_error + ) + + res_conn = get_market_data("mock_db_path", req) + assert res_conn.bars.empty + assert res_conn.message == "Network error or connection timeout" diff --git a/tests/test_storage_database.py b/tests/test_storage_database.py index d513008..9a25ac3 100644 --- a/tests/test_storage_database.py +++ b/tests/test_storage_database.py @@ -1,8 +1,13 @@ from datetime import date - +import pytest import duckdb - -from argus.domain.internal_models import DataSource, Instrument, PriceBar +import pandas as pd +from argus.domain.internal_models import ( + DataSource, + Instrument, + MarketDataRequest, + MarketDataResponse, +) from argus.storage.database import ( initialize_database, insert_price_bar, @@ -10,26 +15,16 @@ ) -def test_initialize_database_creates_required_tables(tmp_path): - db = tmp_path / "test.duckdb" - - initialize_database(db) - connection = duckdb.connect(db) - tables = connection.execute("SHOW TABLES;").fetchall() - connection.close() - table_names = {row[0] for row in tables} - - assert "data_sources" in table_names - assert "instruments" in table_names - assert "price_bars" in table_names - - -def test_data_is_inserted(tmp_path): - source = DataSource( +@pytest.fixture +def sample_source(): + return DataSource( name="Yahoo", provider_kind="yfinance_api", requires_api_key=False ) - instrument = Instrument( + +@pytest.fixture +def sample_instrument(): + return Instrument( symbol="EUR/USD", name="EUR - USD Rate", asset_class="fx", @@ -37,25 +32,52 @@ def test_data_is_inserted(tmp_path): quote_currency="USD", ) - pricebar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), - timeframe="1d", - close=1.89, + +@pytest.fixture +def sample_response(sample_source, sample_instrument): + test_bar = { + "timestamp": date(2026, 1, 1), + "open": None, + "high": None, + "low": None, + "close": 1.89, + "adjusted_close": None, + "volume": None, + } + return MarketDataResponse( + source=sample_source, + instrument=sample_instrument, + bars=pd.DataFrame(test_bar, index=[0]), ) + +@pytest.fixture +def db_path(tmp_path): db = tmp_path / "test.duckdb" initialize_database(db) - insert_price_bar(db, pricebar) - connection = duckdb.connect(db) + return db + + +def test_initialize_database_creates_required_tables(db_path): + connection = duckdb.connect(str(db_path)) + tables = connection.execute("SHOW TABLES;").fetchall() + connection.close() + table_names = {row[0] for row in tables} + + assert "data_sources" in table_names + assert "instruments" in table_names + assert "price_bars" in table_names + + +def test_data_is_inserted(db_path, sample_response) -> None: + insert_price_bar(db_path, sample_response) + + connection = duckdb.connect(str(db_path)) instrument_count = connection.execute( "SELECT COUNT(*) FROM instruments;" ).fetchone() - source_count = connection.execute("SELECT COUNT(*) FROM data_sources;").fetchone() - price_bar_count = connection.execute("SELECT COUNT(*) FROM price_bars;").fetchone() assert instrument_count is not None @@ -66,157 +88,71 @@ def test_data_is_inserted(tmp_path): assert price_bar_count[0] == 1 -def test_fx_has_correct_format(tmp_path): - source = DataSource( - name="Yahoo", provider_kind="yfinance_api", requires_api_key=False - ) +def test_fx_has_correct_format(db_path, sample_response) -> None: + insert_price_bar(db_path, sample_response) - instrument = Instrument( - symbol="EUR/USD", - name="EUR - USD Rate", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) - - pricebar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), - timeframe="1d", - close=1.89, - ) - - db = tmp_path / "test.duckdb" - initialize_database(db) - insert_price_bar(db, pricebar) - connection = duckdb.connect(db) - - price_bar_fx = connection.execute("SELECT * FROM price_bars;").fetchone() - connection.close() + connection = duckdb.connect(str(db_path)) + try: + price_bar_fx = connection.execute("SELECT * FROM price_bars;").fetchone() + finally: + connection.close() assert price_bar_fx is not None assert price_bar_fx[0] == 1 assert price_bar_fx[1] == 1 assert price_bar_fx[2] == 1 assert price_bar_fx[3] == date(2026, 1, 1) - assert price_bar_fx[4] == "1d" - assert price_bar_fx[5] == 1.89 + assert price_bar_fx[4] == 1.89 + assert price_bar_fx[5] is None assert price_bar_fx[6] is None assert price_bar_fx[7] is None assert price_bar_fx[8] is None assert price_bar_fx[9] is None - assert price_bar_fx[10] is None - -def test_duplicates_are_ignored(tmp_path): - source = DataSource( - name="Yahoo", provider_kind="yfinance_api", requires_api_key=False - ) - - instrument = Instrument( - symbol="EUR/USD", - name="EUR - USD Rate", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) - pricebar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), - timeframe="1d", - close=1.89, - ) +def test_duplicates_are_ignored(db_path, sample_response) -> None: + insert_price_bar(db_path, sample_response) + insert_price_bar(db_path, sample_response) # Erneuter Insert des Duplikats - db = tmp_path / "test.duckdb" - initialize_database(db) - insert_price_bar(db, pricebar) - insert_price_bar(db, pricebar) - connection = duckdb.connect(db) - count = connection.execute("SELECT COUNT(*) FROM price_bars;").fetchone() + connection = duckdb.connect(str(db_path)) + try: + count = connection.execute("SELECT COUNT(*) FROM price_bars;").fetchone() + finally: + connection.close() assert count is not None assert count[0] == 1 -def test_read_price_bars_returns_matching_data(tmp_path): - source = DataSource( - name="Yahoo", - provider_kind="yfinance_api", - requires_api_key=False, - ) - - instrument = Instrument( - symbol="EUR/USD", - name="EUR - USD Rate", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) - - pricebar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), +def test_read_price_bars_returns_matching_data( + db_path, sample_source, sample_instrument, sample_response +) -> None: + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, timeframe="1d", - close=1.89, + start=date(2026, 1, 1), + end=date(2026, 1, 1), ) + insert_price_bar(db_path, sample_response) - db = tmp_path / "test.duckdb" - initialize_database(db) - insert_price_bar(db, pricebar) - - result = read_price_bars( - db=db, - source=source, - instrument=instrument, - start_date=date(2026, 1, 1), - end_date=date(2026, 1, 31), - ) + result = read_price_bars(db_path, req) - assert result.empty is False assert len(result) == 1 - assert result.iloc[0]["source_name"] == "Yahoo" - assert result.iloc[0]["instrument_symbol"] == "EUR/USD" - assert result.iloc[0]["timeframe"] == "1d" assert result.iloc[0]["close"] == 1.89 -def test_read_price_bars_returns_empty_dataframe_for_missing_range(tmp_path): - source = DataSource( - name="Yahoo", - provider_kind="yfinance_api", - requires_api_key=False, - ) - - instrument = Instrument( - symbol="EUR/USD", - name="EUR - USD Rate", - asset_class="fx", - base_currency="EUR", - quote_currency="USD", - ) - - pricebar = PriceBar( - source=source, - instrument=instrument, - timestamp=date(2026, 1, 1), +def test_read_price_bars_returns_empty_dataframe_for_missing_range( + db_path, sample_source, sample_instrument +) -> None: + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, timeframe="1d", - close=1.89, + start=date(2026, 1, 1), + end=date(2026, 1, 1), ) - db = tmp_path / "test.duckdb" - initialize_database(db) - insert_price_bar(db, pricebar) - - result = read_price_bars( - db=db, - source=source, - instrument=instrument, - start_date=date(2027, 1, 1), - end_date=date(2027, 1, 31), - ) + result = read_price_bars(db_path, req) assert result.empty is True diff --git a/tests/test_timeseries_service.py b/tests/test_timeseries_service.py deleted file mode 100644 index cd5c97a..0000000 --- a/tests/test_timeseries_service.py +++ /dev/null @@ -1,36 +0,0 @@ -import pandas as pd -import pandas.testing as pdt -import numpy as np -from argus.services.timeseries_service import prepare_trend_analysis - - -def test_get_a_full_timeseries(): - test_curr = "EURUSD=X" - test_start = "2024-01-01" - test_end = "2024-01-04" - test_interval = "1d" - - expect_result = { - "date": ["2024-01-01", "2024-01-02", "2024-01-03"], - "rate": [1.1055831909179688, 1.1038745641708374, 1.0941756963729858], - "daily_pct_change": [np.nan, -0.1545452898675692, -0.8786204622023064], - "roll_avg": [1.1055831909179688, 1.104728877544403, 1.101211150487264], - } - expect_dict = { - "min_date": ["2024-01-03 00:00:00"], - "min_rate": [1.0941756963729858], - "max_date": ["2024-01-01 00:00:00"], - "max_rate": [1.1055831909179688], - } - result = prepare_trend_analysis(test_curr, test_start, test_end, test_interval) - - assert result is not None - - result_df, result_dict = result - result_df["date"] = result_df["date"].astype("str") - result_dict["min_date"] = [str(result_dict["min_date"][0])] - result_dict["max_date"] = [str(result_dict["max_date"][0])] - expect_df = pd.DataFrame(expect_result) - - pdt.assert_frame_equal(result_df, expect_df) - assert result_dict == expect_dict diff --git a/tests/test_trend_analysis_service.py b/tests/test_trend_analysis_service.py new file mode 100644 index 0000000..84f8ccd --- /dev/null +++ b/tests/test_trend_analysis_service.py @@ -0,0 +1,95 @@ +""" +import pytest +import pandas as pd +from datetime import date +from matplotlib.figure import Figure +from argus.domain.internal_models import ( + DataSource, + Instrument, + MarketDataRequest, + MarketDataResponse, +) +from argus.services.trend_analysis_service import prepare_trend_analysis +from argus.services.market_data_service import get_market_data +from argus.storage.database import initialize_database + + +@pytest.fixture +def sample_source(): + return DataSource( + name="Yahoo", provider_kind="yfinance_api", requires_api_key=False + ) + + +@pytest.fixture +def sample_instrument(): + return Instrument( + symbol="EUR/USD", + name="EUR - USD Rate", + asset_class="fx", + base_currency="EUR", + quote_currency="USD", + ) + + +@pytest.fixture +def sample_response(sample_source, sample_instrument): + test_bar = { + "timestamp": [date(2024, 1, 1), date(2024, 1, 2), date(2024, 1, 3)], + "open": [None, None, None], + "high": [None, None, None], + "low": [None, None, None], + "close": [1.10, 1.12, 1.11], + "adjusted_close": [None, None, None], + "volume": [None, None, None], + } + return MarketDataResponse( + source=sample_source, instrument=sample_instrument, bars=pd.DataFrame(test_bar) + ) + + +def test_prepare_trend_analysis_success( + sample_source, sample_instrument, sample_response, monkeypatch +): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) + + monkeypatch.setattr( + "argus.services.trend_analysis_service.get_market_data", + lambda db, r: sample_response, + ) + + res = prepare_trend_analysis("mock_db_path", req) + assert isinstance(res, Figure) + + +def test_prepare_trend_analysis_failure(sample_source, sample_instrument, monkeypatch): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) + + error_response = MarketDataResponse( + source=sample_source, + instrument=sample_instrument, + bars=pd.DataFrame(), + message="Quote not found or no data available for symbol", + ) + + monkeypatch.setattr( + "argus.services.trend_analysis_service.get_market_data", + lambda db, r: error_response, + ) + + res = prepare_trend_analysis("mock_db_path", req) + assert isinstance(res, str) + assert res == "Quote not found or no data available for symbol" +""" diff --git a/tests/test_yfinance_client.py b/tests/test_yfinance_client.py index 6201b19..419d999 100644 --- a/tests/test_yfinance_client.py +++ b/tests/test_yfinance_client.py @@ -1,80 +1,118 @@ from argus.clients.yfinance_client import get_timeseries +from argus.domain.internal_models import DataSource, Instrument, MarketDataRequest +from datetime import date +import pytest import pandas as pd import pandas.testing as pdt -def test_get_dataframe(monkeypatch): +@pytest.fixture +def sample_source(): + return DataSource( + name="Yahoo", provider_kind="yfinance_api", requires_api_key=False + ) + + +@pytest.fixture +def sample_instrument(): + return Instrument( + symbol="EUR/USD", + name="EUR - USD Rate", + asset_class="fx", + base_currency="EUR", + quote_currency="USD", + ) + + +def test_get_dataframe(monkeypatch, sample_source, sample_instrument): test_resp = pd.DataFrame( { + "Open": [None, None, None], + "High": [None, None, None], + "Low": [None, None, None], "Close": [1.105583, 1.103875, 1.094176], + "Adj Close": [None, None, None], + "Volume": [None, None, None], }, index=pd.to_datetime(["2024-01-01", "2024-01-02", "2024-01-03"]), ) test_resp.index.name = "Date" - test_curr = "EURUSD=X" - test_start = "2024-01-01" - test_end = "2024-01-04" - test_interval = "1d" + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) def fake_yfinance_download(*args, **kwargs): return test_resp monkeypatch.setattr("yfinance.download", fake_yfinance_download) - - result = get_timeseries(test_curr, test_start, test_end, test_interval) + resp = get_timeseries(req) expected = pd.DataFrame( { - "date": pd.to_datetime(["2024-01-01", "2024-01-02", "2024-01-03"]), - "rate": [1.105583, 1.103875, 1.094176], + "timestamp": pd.to_datetime(["2024-01-01", "2024-01-02", "2024-01-03"]), + "open": [None, None, None], + "high": [None, None, None], + "low": [None, None, None], + "close": [1.105583, 1.103875, 1.094176], + "adjusted_close": [None, None, None], + "volume": [None, None, None], } ) - assert result is not None - pdt.assert_frame_equal(result, expected) + assert resp is not None + pdt.assert_frame_equal(resp, expected) -def test_get_none(monkeypatch): - test_curr = "EURUSD=X" - test_start = "2024-01-01" - test_end = "2024-01-04" - test_interval = "1d" - - def fake_yfinance_download(*args, **kwargs): - return None - monkeypatch.setattr("yfinance.download", fake_yfinance_download) +def test_client_network_error(monkeypatch, sample_source, sample_instrument): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) - result = get_timeseries(test_curr, test_start, test_end, test_interval) - assert result is None + def mock_crash(*args, **kwargs): + raise Exception() + monkeypatch.setattr("yfinance.download", mock_crash) -def test_get_empty_frame(monkeypatch): - test_curr = "EURUSD=X" - test_start = "2024-01-01" - test_end = "2024-01-01" - test_interval = "1d" + with pytest.raises(ConnectionError, match="Network error or connection timeout"): + get_timeseries(req) - def fake_yfinance_download(*args, **kwargs): - return pd.DataFrame() - monkeypatch.setattr("yfinance.download", fake_yfinance_download) +def test_client_invalid_response(monkeypatch, sample_source, sample_instrument): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) - result = get_timeseries(test_curr, test_start, test_end, test_interval) - assert result is None + monkeypatch.setattr("yfinance.download", lambda *args, **kwargs: None) + with pytest.raises( + ConnectionError, match="Yahoo Finance API returned an invalid response" + ): + get_timeseries(req) -def test_error_raise(monkeypatch): - test_curr = "EURUSD=X" - # start date is inclusiv and end date is exclusiv - the range 2024-01-01-2024-01-01 is not possible - test_start = "2024-01-04" - test_end = "2024-01-02" - test_interval = "1d" - def fake_yfinance_download( - tickers=test_curr, start=test_start, end=test_end, interval=test_interval - ): - raise Exception("fake yfinance error") +def test_client_quote_not_found(monkeypatch, sample_source, sample_instrument): + req = MarketDataRequest( + source=sample_source, + instrument=sample_instrument, + timeframe="1d", + start=date(2024, 1, 1), + end=date(2024, 1, 4), + ) - monkeypatch.setattr("yfinance.download", fake_yfinance_download) + monkeypatch.setattr("yfinance.download", lambda *args, **kwargs: pd.DataFrame()) - result = get_timeseries(test_curr, test_start, test_end, test_interval) - assert result is None + with pytest.raises( + ValueError, match="Quote not found or no data available for symbol" + ): + get_timeseries(req)