- GuruFinance Insights
- Posts
- From Finance Papers to Trading Algorithms: An Automated Approach
From Finance Papers to Trading Algorithms: An Automated Approach
Integrated Extraction and Coding of Alpha Research
Today’s Fastest Growing Company Might Surprise You
🚨 No, it's not the publicly traded tech giant you might expect… Meet $MODE, the disruptor turning phones into potential income generators.
Mode saw 32,481% revenue growth, ranking them the #1 software company on Deloitte’s 2023 fastest-growing companies list.
📲 They’re pioneering "Privatized Universal Basic Income" powered by technology — not government, and their EarnPhone, has already helped consumers earn over $325M!
Their pre-IPO offering is live at just $0.26/share – don’t miss it.
*Mode Mobile recently received their ticker reservation with Nasdaq ($MODE), indicating an intent to IPO in the next 24 months. An intent to IPO is no guarantee that an actual IPO will occur.
*The Deloitte rankings are based on submitted applications and public company database research, with winners selected based on their fiscal-year revenue growth percentage over a three-year period.
*Please read the offering circular and related risks at invest.modemobile.com.
Exciting News: Paid Subscriptions Have Launched! 🚀
On September 1, we officially rolled out our new paid subscription plans at GuruFinance Insights, offering you the chance to take your investing journey to the next level! Whether you're just starting or are a seasoned trader, these plans are packed with exclusive trading strategies, in-depth research paper analysis, ad-free content, monthly AMAsessions, coding tutorials for automating trading strategies, and much more.
Our three tailored plans—Starter Investor, Pro Trader, and Elite Investor—provide a range of valuable tools and personalized support to suit different needs and goals. Don’t miss this opportunity to get real-time trade alerts, access to masterclasses, one-on-one strategy consultations, and be part of our private community group. Click here to explore the plans and see how becoming a premium member can elevate your investment strategy!
Check Out Latest Premium Articles
In this article, we explore a novel AI-assisted workflow designed to automatically generate backtest code from quantitative finance articles. Quantitative finance relies heavily on data-driven strategies derived from extensive research. However, translating insights from dense academic papers into executable trading algorithms can be time-consuming and error-prone. While creating a production-ready solution may not be immediately achievable, our goal is to produce boilerplate code for QuantConnect within seconds. This initial code can help determine whether an article presents a promising strategy that warrants further examination or should be disregarded.
We will detail how to run this code (available on GitHub) and present our initial tests. For foundational principles, this article complements my previous work, “LLM Pair-Programming in Algorithmic Trading.” Notably, our approach utilizes a linear workflow, temporarily setting aside the dual-agent architecture. This tool not only extracts and summarizes key trading strategies and risk management techniques from PDF articles but also generates syntactically correct QuantConnect Python code, complete with syntax highlighting for easy review.
Preliminary tests indicate that the generated code is error-free, although achieving 100% accuracy across all tests has not yet been realized. The NLP component of the tool employs techniques, which can be further refined. Nevertheless, I chose to release the code quickly as a proof of concept in this rapidly evolving field. The project is ongoing and will be refined over time.
How It Works
The core of this automation lies in the ArticleProcessor
class, which orchestrates the entire workflow from PDF extraction to code generation. Here's a breakdown of its functionality:
1. PDF Text Extraction with pdfplumber
The journey begins by loading and extracting text from a PDF file using the pdfplumber
library. Unlike other PDF parsing tools, pdfplumber
offers superior accuracy, especially with complex layouts common in academic articles.
def load_pdf(self, pdf_path: str) -> str:
try:
text = ""
with pdfplumber.open(pdf_path) as pdf:
for page in pdf.pages:
page_text = page.extract_text()
if page_text:
text += page_text + "\n"
logging.info("PDF loaded successfully.")
return text
except Exception as e:
logging.error(f"Failed to load PDF: {e}")
return ""
2. Text Preprocessing
Once the text is extracted, it’s essential to clean and preprocess it. This involves removing URLs, headers, footers, standalone numbers (like page numbers), and other irrelevant content using regular expressions.
def preprocess_text(self, text: str) -> str:
try:
text = re.sub(r'https?://\S+', '', text)
text = re.sub(r'Electronic copy available at: .*', '', text)
text = re.sub(r'^\d+\s*$', '', text, flags=re.MULTILINE)
text = re.sub(r'\n+', '\n', text)
text = re.sub(r'^\s*(Author|Title|Abstract)\s*$', '', text, flags=re.MULTILINE | re.IGNORECASE)
text = text.strip()
logging.info("Text preprocessed successfully.")
return text
except Exception as e:
logging.error(f"Failed to preprocess text: {e}")
return ""
3. Heading Detection with SpaCy
Understanding the structure of the article is crucial. Using SpaCy’s NLP capabilities, the tool identifies section headings based on heuristics like sentence length and title casing. This segmentation aids in organizing content for more effective analysis.
def detect_headings(self, text: str) -> List[str]:
try:
doc = self.nlp(text)
headings = []
for sent in doc.sents:
sent_text = sent.text.strip()
if len(sent_text.split()) < 10 and sent_text.istitle():
headings.append(sent_text)
logging.info(f"Detected {len(headings)} headings.")
return headings
except Exception as e:
logging.error(f"Failed to detect headings: {e}")
return []
4. Section Splitting and Keyword Analysis
With the headings identified, the text is split into respective sections. The tool then performs a keyword analysis to categorize sentences into trading_signal
and risk_management
. This categorization is based on predefined lists of relevant keywords, ensuring that only pertinent information is processed further.
def keyword_analysis(self, sections: Dict[str, str]) -> Dict[str, List[str]]:
keyword_map = defaultdict(list)
risk_management_keywords = [
"drawdown", "volatility", "reduce", "limit", "risk", "risk-adjusted",
"maximal drawdown", "market volatility", "bear markets", "stability",
"sidestep", "reduce drawdown", "stop-loss", "position sizing", "hedging"
]
trading_signal_keywords = [
"buy", "sell", "signal", "indicator", "trend", "SMA", "moving average",
"momentum", "RSI", "MACD", "bollinger bands", "Rachev ratio", "stay long",
"exit", "market timing", "yield curve", "recession", "unemployment",
"housing starts", "Treasuries", "economic indicator"
]
irrelevant_patterns = [
r'figure \d+',
r'\[\d+\]',
r'\(.*?\)',
r'chart',
r'\bfigure\b',
r'performance chart',
r'\d{4}-\d{4}',
r'^\s*$'
]
processed_sentences = set()
for section, content in sections.items():
doc = self.nlp(content)
for sent in doc.sents:
sent_text = sent.text.lower().strip()
if any(re.search(pattern, sent_text) for pattern in irrelevant_patterns):
continue
if sent_text in processed_sentences:
continue
processed_sentences.add(sent_text)
if any(kw in sent_text for kw in trading_signal_keywords):
keyword_map['trading_signal'].append(sent.text.strip())
elif any(kw in sent_text for kw in risk_management_keywords):
keyword_map['risk_management'].append(sent.text.strip())
for category, sentences in keyword_map.items():
unique_sentences = list(set(sentences))
keyword_map[category] = sorted(unique_sentences, key=len)
logging.info("Keyword analysis completed.")
return keyword_map
5. Generating Summaries and QuantConnect Code with OpenAI’s GPT-4
Harnessing the capabilities of OpenAI’s GPT-4, the tool generates concise summaries of the extracted strategies and risk management techniques. More impressively, it transforms these insights into fully functional QuantConnect Python algorithms, ensuring they adhere to best practices and syntactic correctness.
def generate_qc_code(self, extracted_data: Dict[str, List[str]]) -> str:
trading_signals = '\n'.join(extracted_data.get('trading_signal', []))
risk_management = '\n'.join(extracted_data.get('risk_management', []))
prompt = f"""
You are an expert QuantConnect algorithm developer. Convert the following trading strategy and risk management descriptions into a complete, error-free QuantConnect Python algorithm.
### Trading Strategy:
{trading_signals}
### Risk Management:
{risk_management}
### Requirements:
1. **Initialize Method**:
- Set the start and end dates.
- Set the initial cash.
- Define the universe selection logic.
- Initialize required indicators.
2. **OnData Method**:
- Implement buy/sell logic based on indicators.
- Ensure indicators are updated correctly.
3. **Risk Management**:
- Implement drawdown limit of 15%.
- Apply position sizing or stop-loss mechanisms as described.
4. **Ensure Compliance**:
- Use only QuantConnect's supported indicators and methods.
- The code must be syntactically correct and free of errors.
### Example Structure:
```python
from AlgorithmImports import *
class MyAlgorithm(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2020, 1, 1)
self.SetEndDate(2023, 1, 1)
self.SetCash(100000)
# Define universe, indicators, etc.
def OnData(self, data):
# Trading logic
def OnEndOfDay(self):
# Risk management
```
### Generated Code:
```
# The LLM will generate the code after this line
```
"""
try:
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a helpful assistant specialized in generating QuantConnect algorithms in Python."},
{"role": "user", "content": prompt}
],
max_tokens=2500,
temperature=0.3,
n=1
)
generated_code = response['choices'][0]['message']['content'].strip()
code_match = re.search(r'```python(.*?)```', generated_code, re.DOTALL)
if code_match:
generated_code = code_match.group(1).strip()
logging.info("Code generated by LLM.")
return generated_code
except Exception as e:
logging.error(f"Failed to generate code: {e}")
return ""
6. Displaying Results with Tkinter and Pygments
To provide a user-friendly experience, the tool uses Tkinter to display the article summary and the generated code in separate windows. The Pygments
library enhances readability by adding syntax highlighting to the Python code.
def display_summary_and_code(self, summary: str, code: str):
# Create the main Tkinter root
root = tk.Tk()
root.withdraw() # Hide the root window
# Summary Window
summary_window = tk.Toplevel()
summary_window.title("Article Summary")
summary_window.geometry("800x600")
summary_text = scrolledtext.ScrolledText(summary_window, wrap=tk.WORD, font=("Arial", 12))
summary_text.pack(expand=True, fill='both')
summary_text.insert(tk.END, summary)
summary_text.configure(state='disabled') # Make it read-only
# Code Window
code_window = tk.Toplevel()
code_window.title("Generated QuantConnect Code")
code_window.geometry("1000x800")
code_text = scrolledtext.ScrolledText(code_window, wrap=tk.NONE, font=("Consolas", 12), bg="#2B2B2B", fg="#F8F8F2")
code_text.pack(expand=True, fill='both')
# Apply syntax highlighting
self.apply_syntax_highlighting(code, code_text)
code_text.configure(state='disabled') # Make it read-only.
# Start the Tkinter event loop
root.mainloop()
7. First test
We use the article ‘Avoid bear markets with a market timing strategy’ of L. Durian and R. Vojtko from Quantpedia. The output is reproduced below :

A view of the code and summary generated.
The code generated during the first run generates the following backtest :

Backtest of the code generated — Powered by QuantConnect
We can see that the algorithm is idle in 2020, as expected for a strategy avoiding the Covid bear market. The article seems indeed managing the avoidance of bear market and is definitely worth further examination.
Second test (3 Oct 2024) after OOP refactoring
There are still marginal coding error using chatgpt-4o-latest LLM. From the article ‘Composite Seasonal Equity Trading Strategy’ of the same author, I obtain the following backtest :

Composite seasonal equity trading strategy — Powered by QuantConnect
A backtest starting 1st of January 2024:

YTD backtest of the same strategy — Powered by QuantConnect
A view of the GUI :

7. Black-box approach
Once the article-to-code workflow was fully formalized in Python, it became tempting to design a customized GPT based on these principles. The primary disadvantage is that, unlike hard coding in Python, there is no iteration possible within the black box. However, a significant advantage is that if a logic or compilation error occurs, the user can engage in a chat with the GPT to debug the code effectively.
Here is the result:

Pragmatic asset allocation model for semi-active investor — Powered by QuantConnect
The initial code generated is error-free; however, it is essential to thoroughly verify the trading logic against the findings presented in the original paper.
8. Tests performed on 7th October 2024
QuantPedia is a research platform that curates and analyzes academic studies on quantitative trading strategies. It provides users with insights into various trading models, offering detailed explanations, backtests, and performance metrics. The platform is designed to help investors discover and implement evidence-based investment strategies.
For this new test, I selected three free strategies available on the platform (full references are provided at the end of the article) and asked the program to generate backtesting code. My primary focus is on ensuring the code compiles without errors and executes trades; I am not yet concerned with optimizing strategy performance.
To clarify, the goal of this project is not to design a fully autonomous AI quant. Rather, it aims to develop an AI assistant capable of quickly drafting baseline code, which can then be refined by human quantitative expertise.
The first test involved the article on Relative Strength Strategies for Investing. The generated code was error-free, and the algorithm executed trades as specified in the backtest. I encourage readers to compare the results with the backtest provided by QuantPedia. While the automated strategy underperformed significantly compared to the QuantPedia report, this does not imply an issue with QuantPedia’s findings or the strategy itself. It simply indicates that further analysis by a quant is needed to refine the strategy and uncover its alpha potential.

Test A —From Relative Strength Strategies for Investing — Powered by QuantConnect
The second test was conducted using the article Pairs Trading: Performance of a Relative Value Arbitrage Rule. Once again, the code was generated without errors, and several trades were executed. The backtest results are provided below. Similar to the previous test, the backtest underperforms significantly compared to the original analysis by QuantPedia. This highlights the need for further refinement and deeper quantitative analysis to match the performance metrics outlined in the QuantPedia study.

Test B — Pair tradings: performance of a relative arbitrage value rule. — Powered by QuantConnect
The final test was based on the article The Volatility Effect: Lower Risk without Lower Return. The initial code contained a compilation error for QuantConnect:
“Trying to retrieve an element from a collection using a key that does not exist throws a KeyError exception. To prevent this, ensure the key exists in the collection and/or that the collection is not empty.”
This occurred at:
sorted_by_volatility = sorted(fine, key=lambda x: self.volatility[x.Symbol].Current.Value)
~~~~~~~~~~~~~~~^^^^^^^^^^
While this issue was easy to fix, the algorithm did not enter any trades. The underlying logic needs further investigation, as the strategy is not functioning as intended. In this case, correcting the code alone isn’t enough; a deeper review of the article’s strategy is required to properly implement and test it from the ground up.
9. Quant Coder : Seamlessly Integrating Research and Code Generation
Building on previous developments, the workflow has been significantly enhanced with a tool that integrates article research, summarization, and code generation. This streamlined process allows users to search for relevant research articles, download them, generate summaries, and infer algorithmic trading code from the summarized content — all within a single interface.
Using the CrossRef API, as outlined in the article “Searching Academic Articles with Python and the CrossRef API,” the workflow now includes a user-friendly graphical interface (GUI) featuring three core functions:
Search: Users can search for academic articles based on a query, with the option to return a specified number of articles.
Summarize: On request, the selected article is downloaded and summarized using advanced natural language processing (NLP) techniques.
Code: From the generated summary, QuantConnect-compatible Python code is automatically inferred, providing a seamless transition from research to implementation.
Each article retrieved in the search results can be accessed through the intuitive interface by double-clicking on its title, simplifying the process of exploration and selection.

The Search function of the Quant Coder
For example, an article can be accessed by simply double-clicking its title in the search results.

One article found by the search function.
The generated summary is automatically saved for future reference.

The code is generated based on the selected summary (from a text file).

The beta version of QuantCoder is available on a dedicated branch of the GitHub repository.
Conclusion
Automating the transformation of quantitative finance research into executable trading algorithms can greatly enhance efficiency, bridging the gap between research and implementation. By leveraging powerful libraries like pdfplumber, SpaCy, OpenAI’s GPT-4, Tkinter, and Pygments, this Python-based tool offers a streamlined solution that simplifies the traditionally complex process of translating research into actionable code.
The value of this tool can be assessed in two key ways. First, the generated code should ideally be error-free from the start. Based on tests conducted with six articles, the tool has demonstrated a high degree of reliability, with an estimated success rate of 70% in generating error-free code in the latest version published on GitHub. Second, the code must successfully enter trades before assessing the overall merit of the strategy. On this criterion, the success rate is even higher, around 80%. This suggests that, with the latest GPT-4 model and proper parameters, roughly 75% of the generated code is immediately viable for implementation, providing a solid foundation for further refinement by quantitative experts. I have retained certain proprietary elements, such as the specific wording of prompts and the number of iterations used. However, the open-source code is freely available for others to experiment with and adapt to their own requirements. For me, this tool has become a key part of my development toolkit, and I’m already considering what the next tool to design will be.
References
Ďurian, Ladislav and Vojtko, Radovan, Avoid Equity Bear Markets with a Market Timing Strategy (March 23, 2023). Available at SSRN: https://ssrn.com/abstract=4397638 or http://dx.doi.org/10.2139/ssrn.4397638
Vojtko, Radovan and Padyšák, Matúš, Composite Seasonal Equity Trading Strategy (October 25, 2019). Available at SSRN: https://ssrn.com/abstract=3475660 or http://dx.doi.org/10.2139/ssrn.3475660
Vojtko, Radovan and Javorská, Juliána, Pragmatic Asset Allocation Model for Semi-Active Investors (February 3, 2024). Available at SSRN: https://ssrn.com/abstract=4715415 or http://dx.doi.org/10.2139/ssrn.4715415
Faber, M. (2010). Relative Strength Strategies for Investing. SSRN. Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1585517.
Gatev, E. (2006). Pairs Trading: Performance of a Relative Value Arbitrage Rule. SSRN. Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=141615.
Blitz, D. C., & van Vliet, P. (2007). The Volatility Effect: Lower Risk Without Lower Return. SSRN. Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=980865.
7. Code of the project :
8. Link to the customized GPT:
9. Prototyping ideas using MyShell :
10. QuantConnect : The World’s leading cloud quant platform
Happy coding and trading !