Coinrule vs Botsfolio: Do Rules Tuned on the Past Hold Up?

Jay Sharma · Founder, Botsfolio

Published November 14, 2026

Coinrule's homepage shows what a trading rule might have made. Under the figures sits a sentence most companies in this business would not print: "Simulation only. Figures are illustrative, based on representative patterns, not actual trading results."

Good for Coinrule. In a market full of screenshots, that label is a small act of honesty. This page will not pretend otherwise.

But read the sentence again as a buyer. It tells you that the one number on the page that looks like a result is not one. Which leaves the real question standing. When you write a rule, test it on past prices and like what you see, how much is that worth once the rule meets a market it has never met? There is good research on exactly that. It is the heart of this page. We make Botsfolio, so weigh that. Every source is linked.

What Coinrule is

Coinrule is a rule builder. You write conditions in plain form: if this crypto falls by so much, buy this amount; if it rises by so much, sell. No code. If you'd rather not start from a blank page, there is a template library. The homepage invites you to "Choose Among 350 + Trading Agents" and adds, fairly, that "Every template is a starting point."

The rules run on your own exchange account through an API key. Coinrule is plain about its own role: "Coinrule is not a regulated financial institution and does not provide investment advice."

So the product is a way to automate a strategy. The strategy is yours.

What Coinrule shows about results

Two lines from the homepage carry it.

"Simulation only. Figures are illustrative, based on representative patterns, not actual trading results."

"Any performance figures shown are illustrative only."

We read the site on November 13, 2026 and found no live results: no record of what its templates did after customers switched them on and no figure for how its users have done. To be fair, a rule builder can't easily publish one. Every customer runs something different.

That is the honest position. It has a consequence. The only evidence you will ever have for your rule, before you risk money on it, is a backtest.

The trap in a good backtest

Coinrule gives you a backtester and encourages you to use it. Its backtesting page tells you to "Use validation to find optimal settings" for indicators, stop losses and profit targets.

It sounds like diligence. Try a setting, check the result, try another, keep the best. Here is the problem. Every time you adjust a rule and keep the version that scored higher, you fit it a little more closely to the prices that already happened. Past prices are part pattern and part noise. The pattern might repeat. The noise will not. A rule tuned long enough ends up as a very precise description of last year.

This is not a hunch. It has been measured.

The Quantopian study. Researchers at Quantopian, a platform where the public wrote and tested trading strategies, examined "888 algorithmic trading strategies" that had at least six months of results on new prices after their backtests. They found that the standard backtest scores "offer little value in predicting out of sample performance". Out of sample means on prices the strategy had never seen. And they found the mechanism: "the more backtesting a quant has done for a strategy, the larger the discrepancy between backtest and out-of-sample performance". The harder people worked on a rule, the more it disappointed. (The study covered US equity strategies. We read its abstract as reproduced by Quantpedia. The paper itself is on SSRN.)

In crypto. A 2021 paper in Finance Research Letters tested trading rules in crypto markets while correcting for exactly this kind of tuning. Its finding: "statistically significant positive excess returns are rarely achieved".

A vendor's own test. Altrady, another bot company, tuned a simple trend rule on bitcoin prices from 2020 to 2023 and then ran it on what came after. It reported that the rule "earned 83% a year on the data it was tuned on and 29% a year afterwards". Still positive, in a rising market. But look at the size of the haircut.

The other side. The evidence is not all one way. A page like this should say so. A 2021 study in the Annals of Operations Research tested almost 15,000 technical rules and found profits in every class of rule on the data it examined. Then came the qualifier: "there is no predictability for Bitcoin in the out-of-sample period". Rules can work. The trouble is knowing in advance whether yours is one of them. A backtest you tuned yourself is the weakest possible witness.

We cover the five ways a backtest flatters a strategy, with the tests that catch them, in why trading bots fail live.

What about Coinrule's AI?

AI made building a bot easy. It did not make the idea any better.

Coinrule now leads with AI. Its homepage headline is "Create AI Trading Bots". Its documentation describes the first step like this: "Paste Python, Pine Script, or plain-English." The AI turns the description into a rule. You can backtest it and run it on paper first ("Or start on paper with a virtual $10,000.") before you deploy it.

It also goes a step beyond the chat box. Through a connector called Coinrule MCP, ChatGPT, Claude or Grok can work on your account directly. The pitch: "Turn plain-language prompts into real trades your AI assistant can execute, monitor and manage." The workflow it suggests is the one this page has been describing: "Ask your assistant to run a backtest on historical data before going live, then launch the strategy when you are happy with it."

Two things are worth saying about that.

First, the paper step is good practice. Coinrule deserves credit for putting it up front.

Second, AI makes the trap in the last section easier to fall into, not harder. When a new version of a rule costs one sentence, you can try fifty before lunch. The Quantopian finding was that more backtesting meant a bigger gap between the backtest and what followed. "When you are happy with it" describes the moment the fitting is finished.

What happens when a general AI model makes the calls? We run that test in public. In our battles, Claude and GPT models each trade the same paper money on the same live prices as Botsfolio AI, next to simply holding. Can ChatGPT trade crypto has the longer story.

Botsfolio does not hand you that job. You pick the cryptos and the amount. Botsfolio AI does the research and testing and trades them on your own exchange account. What you get to judge is a live record, not a backtest.

Is Coinrule safe?

On design, yes. From the homepage FAQ: "Coinrule is completely non-custodial, meaning we never have access to your funds. It uses API connections with no withdrawal permissions, so it can only execute trades on your behalf." Your cryptos stay on your exchange.

We looked for reports of a breach or a regulator's action against Coinrule and found none.

What Coinrule costs

There is a free plan, which is a real advantage for someone who wants to learn by doing. On November 13, 2026 the first paid plan was listed at $29.99 a month. Its card also showed "0.02% of traded volume". How a monthly fee weighs on a small account is worked out in what crypto trading bots cost.

A note on comparison pages, this one included

Coinrule publishes comparison pages of its own. The one that sets it against 3Commas and Cryptohopper ends like this: "Ultimately, all of these features combined make Coinrule the best automated trading platform overall."

We would rather not end that way. A page written by one of the two companies can't also be the judge. So this page gives no verdict. It quotes sources you can open, says where Coinrule is the better choice and leaves the call to you.

What you have to do yourself

Writing the rule is the easy part. Coinrule makes it easier still. The hard part is everything the research above describes: testing without fooling yourself, keeping some history aside that you never tune on, counting how many versions you tried and stopping a rule when it stops working. That is a research job. It is a fine hobby and a difficult profession.

How Botsfolio is built

Botsfolio takes that research job on. You pick the cryptos and the amount. You can stop at any time. Botsfolio AI does the research. It generates trade ideas, tests them on years of market data, finds what is working and uses that to trade the cryptos you picked. Most ideas fail its tests and are never used.

Why should you believe testing done by the company that sells the product? You shouldn't, on its word alone. So here is what you can check.

The failures are published. Our methodology page reports the tests the portfolios were put through, including one that fell short of the bar we had set for it in advance. It is still there, with its numbers.

The live record is separate from the backtest. Results are kept as three records: real money, paper trading on live markets and a backtest. They are labelled, dated and never blended. The live ones are the answer to the Quantopian problem, because they are prices the research never saw.

Every live trade is public. Each filled trade goes into a log that is sealed every hour, signed by Botsfolio and anchored in Bitcoin. A seal shows who published a row, when it existed and that it has not changed since.

Here are the three records as of the last daily close:

You can also watch the difference between a plan and a result in real time. In our live battles, general AI models and Botsfolio AI each trade the same paper money on the same prices. Every trade is logged as it happens.

Where Coinrule is the better choice

• You want to build your own strategy. That is what Coinrule is for. Its rule builder needs no code. Botsfolio is not a rule builder.
• You want to learn how automated trading works. A free plan and a template library are a good classroom.
• You have a rule you already trust. If you have tested an idea properly, kept data aside and watched it hold up live, a rule builder is all you need.
• You want your own assistant to do it. Coinrule lets ChatGPT, Claude or Grok build and launch strategies on your account by chat.

If you do build on Coinrule, the research suggests three habits. Decide what counts as a pass before you run the test. Keep a stretch of history that you never tune on. And run the rule on paper, on live prices, before it touches real money.

Side by side

FAQ

Sources

Botsfolio is software. It is not investment advice and it is not tailored to you. Botsfolio has no affiliation with Coinrule. Coinrule is a trademark of its owner, used here only to name the product. Paper trading results are simulated. Trading crypto can lose money. Past performance does not predict future results.