Automate NSE F&O data collection with the BarathGB007/nse-options-data-collector: 37 symbols, option chains, Greeks, IV, OI every 15 minutes in Parquet.
Manual NSE options data collection eats up hours and often produces messy datasets. Traders, quants, and researchers need a reliable way to capture option chains, open interest, Greeks, and implied volatility at regular intervals. The BarathGB007/nse-options-data-collector repository offers a practical open-source solution. It automates NSE F&O data collection for 37 symbols, snapshots every 15 minutes, and stores everything in Parquet for efficient analysis.
In this article, we will break down what this NSE options data collector does, why automated collection matters, how the pipeline works, and how you can use it for real-world research and trading.
India’s derivatives market has grown rapidly over the past decade. The National Stock Exchange (NSE) is consistently among the world’s largest options exchanges by volume. Yet despite the market’s scale, high-quality historical and intraday option chain data remains surprisingly hard to access. The NSE website provides real-time snapshots, but it does not offer a clean, queryable record of expired strikes, changing open interest, or historical implied volatility.
If you are building a backtest, an options screener, or a trading dashboard, you need to collect data yourself. Manual collection is painful. You might copy and paste web pages, splice together CSV exports, or write one-off scripts that break whenever the website changes. The result is often inconsistent timestamps, duplicated rows, and missing columns.
An automated pipeline solves these problems by collecting data on a fixed schedule, normalizing it, and storing it in a consistent format. Automation also enables reproducibility. When your data collection is automated, you can reproduce research results months later. This is critical for quantitative trading and academic studies.
The BarathGB007/nse-options-data-collector is built specifically for the Indian options market. According to the repository, it captures the following for NSE F&O instruments:
The pipeline covers exactly 37 symbols. That includes liquid indices and stocks in the F&O segment, making it useful for most retail and semiprofessional options strategies.
Data is collected every 15 minutes. This frequency gives you a good balance between granularity and storage overhead. Intraday patterns, such as morning volatility spikes or afternoon unwinding, remain visible. Yet you do not end up with millions of ultrafine tick rows.
For a 6.25-hour trading day, that equates to about 25 snapshots per symbol per day. Across 37 symbols, you can generate roughly 925 structured snapshots daily. Those numbers are manageable for a small VPS.
Another useful feature is the inclusion of global cues. These are external market signals—such as global index levels or currency moves—that provide context for local options activity. For researchers, this simplifies the process of aligning NSE data with global sentiment. It also reduces the number of data sources you need to integrate manually.
The repository’s architecture follows a common data engineering pattern: extract, transform, load (ETL). At a scheduled interval, the collector pulls option chain data from NSE’s public endpoints, enriches it with derived metrics, and writes it to storage.
The workflow typically looks like this:
Because the collector is designed to run unattended, a VPS is the recommended deployment target. You set it up once, and it becomes a persistent market data recorder. This is far more reliable than running the script on a local laptop that might sleep, reboot, or lose internet connectivity.
Automated collection reduces human error, but you should still monitor the pipeline. Network timeouts, NSE endpoint changes, and public holiday schedules can create gaps. The repository’s design makes this monitoring easier because all data lands in the same storage format and can be audited for missing timestamps.
If you deploy it yourself, add a simple health check that alerts you when no new file appears during market hours. That small step ensures your historical dataset remains complete.
The choice of Parquet as the storage format is a major advantage. Parquet is a columnar storage format that compresses data efficiently and supports fast analytical queries. It was originally built for distributed systems like Hadoop, but today it is the default format for many Python data tools.
Why does this matter for NSE options data?
To illustrate, a simple query in Python might look like this:
import pandas as pd
df = pd.read_parquet('nse_options/')
nifty = df[df['symbol'] == 'NIFTY']
latest = nifty[nifty['timestamp'] == nifty['timestamp'].max()]
print(latest[['strike', 'option_type', 'open_interest', 'implied_volatility']].head())
This example is illustrative, but it shows how Parquet fits into a data analysis workflow. You can load years of history without loading terabytes into memory, especially if you use a query engine like DuckDB.
The data produced by this NSE options data collector opens the door to a variety of applications.
With 15-minute snapshots, you can build a live options dashboard that updates throughout the session. Track OI changes at key strikes, watch IV move with price action, and identify unusual options activity in real time. This is useful for day traders who trade index options or stock options.
Backtesting options strategies requires historical option chain data, including OI and Greeks. With this collector, you can store months of clean data and test strategies like covered calls, iron condors, or bull call spreads with realistic entry assumptions.
Implied volatility is not a single number. It varies by strike and expiry. Researchers can use this data to construct and analyze IV surfaces. These surfaces are essential for relative-value trading and risk management.
Open interest changes are a powerful sentiment signal. For example, rising OI in out-of-the-money put options can indicate hedging demand. The put-call ratio, derived from aggregate OI, is widely used as a contrarian indicator. An automated pipeline allows you to compute these indicators consistently over time.
Quantitative models need granular features. You can create features from the data, such as IV rank, OI momentum, bid-ask spreads, and Greek sensitivities. These features can feed models that predict short-term volatility or price direction.
Those use cases show why automated collection matters. Without the data, even the best model has nothing to learn from.
Automated options data collection is not a niche experiment. It is part of a broader shift toward self-serve market data infrastructure.
Over the last three to five years, retail algo trading has expanded significantly in India. More traders are comfortable running Python scripts and deploying small servers. Open-source projects like this one give them access to institutional-grade data pipelines. The result is a more level playing field between large firms and individual analysts.
In the last two to three years, columnar storage formats like Parquet have become the default for financial data pipelines. Data scientists prefer Parquet over CSV because of speed and storage efficiency. This repository reflects that trend by choosing Parquet instead of storing data in flat files.
Another macro trend is the growing demand for historical options data. As retail options traders become more sophisticated, they want to backtest with accurate historical Greeks, IV, and OI. Automated collectors are essential to fill that gap.
If you decide to deploy the repository, keep these best practices in mind:
The project already includes VPS deployment scripts, which reduces the upfront effort. Use them as a starting point, then customize the configuration for your own watchlist.
The best way to understand the collector is to try it. Clone the GitHub repository, read the instructions, and deploy it on a small cloud server. Start with a single symbol and confirm that snapshots are saved before expanding to all 37 symbols.
Once data starts flowing, explore the Parquet files with pandas or DuckDB. Build a simple dashboard that shows OI changes and IV by strike. This hands-on exercise will give you a foundation for more advanced options analytics.
The repository also demonstrates a clean, reproducible approach to financial data collection. Even if you do not trade options, you can learn a lot from its design patterns, including scheduled job execution, data normalization, and efficient storage.
The BarathGB007/nse-options-data-collector removes a major friction point in Indian options research. It captures rich, structured data for 37 symbols every 15 minutes, including OI, Greeks, IV, bid/ask prices, and EOD snapshots. By storing data in Parquet and supporting VPS deployment, it is practical enough for continuous use.
For traders and researchers, the main takeaway is simple: start building your own options data warehouse. Automate collection, store it efficiently, and you can focus on generating insights instead of scraping web pages. Explore the repository, adapt it to your workflow, and join the growing trend of data-driven options analysis.
It is an open-source Python tool that automates the collection of NSE F&O options data. It captures option chains, Greeks, implied volatility, and open interest for 37 symbols every 15 minutes and stores the data in Parquet format.
The tool captures full option chain details such as strike prices, expiry dates, open interest, trading volumes, bid and ask prices, implied volatility, and Greeks including delta, gamma, theta, and vega.
You can run the collector as a Python script and schedule it with cron (Linux/macOS) or Task Scheduler (Windows). The repository is designed for repeatable 15-minute snapshots, so you only need to set the interval and ensure the script runs continuously during market hours.
The code is free and open-source, so there is no licensing fee. You will need Python, the dependencies listed in the repository, and a reliable internet connection. The tool uses publicly available NSE endpoints, but you should review and comply with NSE's terms of service.
Parquet is a columnar storage format that is more efficient for large financial datasets. It compresses well, preserves data types, and allows faster filtering and aggregation when working with tools like pandas, DuckDB, or Spark.
