Skip to main content
SUBMIT AN ISSUElast edit: Feb 28, 2024

Subnet 9

Name​

Pretrain Subnet

Netuid​

9

Description​

This subnet rewards miners for producing pretrained Foundation-Models on the Falcon Refined Web dataset. It acts like a continuous benchmark whereby miners are rewarded for attaining the best losses on randomly sampled pages of Falcon given a consistent model architecture. The reward mechanism works as follows:

  1. Miners train and periodically host trained model weights linked to their miner key as exampled by the code in neurons/miner.py.
  2. Validators run a continuous eval on the hosted models, performing the validation system outlined in neurons/validator.py and setting weights to the chain based on the performance of each miner on the Falcon dataset.
  3. The chain aggregates weights from all active validators and runs Yuma Consensus to determine the proportion of TAO emission rewarded to miners and validators.

Subnet owner​

https://github.com/unconst/pretrain-subnet/

GitHub​

https://github.com/unconst/pretrain-subnet/