1. Develop a mechanism to represent neural networks architectures (including transformers) as DAGs. 2. Implement hyperparameter mutation and recombination operators. 3. Implement architectural mutation and recombination operators which ensure valid architectures.

The creation of a framework which can create artificial neural networks from DAGs. By training on very small amounts of data we will also confirm the framework’s ability to generate valid architectures using mutation and recombination.

$12,000 USD

success_criteria_1

1. Integrate evaluation and selection mechanisms using standard NLP datasets and fitness metrics. 2. Run initial evolutionary experiments. Try different neural network sizes and dataset sizes to generate data about how these experiments and architectures differ across scale.

An updated framework which is feature-complete for evolutionary runs. The output of the evolutionary experiments including perplexity on next-word prediction and other related evolutionary output metrics.

$8,000 USD

success_criteria_1

1. Develop advanced mutation and recombination operators incorporating biases toward favorable traits such as modularity or sparsity. 2. Run further experiments using these updated operators including creative population initialization (such as portions of the population having varying subsets or magnitudes of these new biases).

An updated framework which includes these advanced variation operators aimed to improve the search of the evolutionary algorithm.. The output of the evolutionary experiments including perplexity on next-word prediction and other related evolutionary output metrics.

$8,000 USD

success_criteria_1

1. Scale up the best-performing architectures for larger-scale training runs. 2. Compare the scaled models’ performance against known transformers of similar scale using training/test metrics and benchmark results. 3. Document findings and release codebase with comprehensive instructions and supporting documentation.

Larger-scale training runs to demonstrate the scalability of the best output model architectures from the evolutionary runs. Also the final codebase and report of our findings over the entire project.

$12,000 USD

success_criteria_1