# Data sources and operational limits

MarketPilot AI includes lightweight adapters for public endpoints. Availability, terms, fields and rate limits can change. A public endpoint is not necessarily licensed for redistribution or commercial use. Verify provider terms before building a commercial service.

| Source | Use in MarketPilot AI | Important limit |
|---|---|---|
| Yahoo Finance chart endpoint | OHLCV bars | Unofficial endpoint; can lag, throttle, revise or change without notice |
| Yahoo Finance RSS | Headline metadata | Headlines/snippets are leads; open the article before stating facts |
| Stooq CSV | Daily OHLCV adapter | Coverage, symbol mapping, delays and use terms vary |
| Nasdaq Trader symbol directories | Symbol universe | Public directory fields, filters and publication timing can change; not a complete investable universe |
| SEC EDGAR data.sec.gov | Company submissions and XBRL facts | Requires a descriptive `User-Agent`; respect SEC fair-access guidance and retrieve only what is needed |
| U.S. House Clerk disclosures | Annual House filing index and PTR PDFs | A filing index is not the contents of the report; transaction disclosures can be delayed and require document-level verification |

## Provider links

- Nasdaq Trader symbol directory definitions: https://www.nasdaqtrader.com/Trader.aspx?id=SymbolDirDefs
- Nasdaq Trader symbol directory lookup/downloads: https://nasdaqtrader.com/Trader.aspx?id=symbollookup
- SEC Developer Resources: https://www.sec.gov/about/developer-resources
- SEC EDGAR APIs: https://www.sec.gov/search-filings/edgar-application-programming-interfaces
- U.S. House Clerk financial disclosure reports: https://disclosures-clerk.house.gov/FinancialDisclosure
- House Committee on Ethics disclosure requirements: https://ethics.house.gov/financial-disclosure/

## Rate and identity practices

- The SEC's developer guidance states a maximum of 10 requests per second and asks automated clients to identify themselves with a declared User-Agent. MarketPilot AI's SEC client requires a local `SEC_USER_AGENT` containing a contact name and email. Use a real address you control; never paste it into chat or commit it.
- The universe scanner defaults to 100 names and includes a pause between public market-data calls. Set a higher cap only after accepting provider terms and the time/rate-limit implications. If any names fail, the response must stay labeled partial.
- Do not use scraping to bypass authentication, paywalls, legal notices, rate limits, or site access controls.

## Market-data caveats

- Online daily commands exclude the current exchange-local date from analysis to avoid a potentially incomplete daily candle; this can leave the latest complete close unused until the next session date. Free data may be delayed and may not contain all trades, quote conditions, corporate actions, halts or auction prints.
- Intraday candles are not tick data. A candle's high and low do not reveal which level traded first; the simulator resolves a same-bar stop/target collision against the trade.
- A stop trigger is not a guaranteed fill price. Gaps, volatility, spread, low liquidity and exchange halts can create materially worse execution.
- A data-quality `PASS` checks only structural conditions implemented by the utility. It does not certify correctness, survivorship-bias-free history, corporate-action adjustment or provider completeness.