How can you test the work of AI agents?
First, we create a virtual environment in which we have full control over all parameters — a so-called deterministic simulation. Then, we introduce elements of randomness so that each run slightly differs from the previous one, making agent training more challenging — for this, we fix the seed. After that, we describe a specific case or scenario, adding details to the environment based on it.
Since we have complete control over the environment, we know the correct responses to all possible situations within it in advance. Next, we develop a set of tests that compare the agent’s actual actions with our expectations. Repeating this entire process many times — say, a hundred — results in a series of tasks for the BitGN PAC1 competition.
The screenshot shows an example of one of the preparatory tasks from Sandbox — the assignments I plan to run next week. They closely resemble ERC3, only the environment is slightly different.
Your, @llm_under_hood 🤗
Created with n8n:
https://cutt.ly/n8n
Created with syllaby:
https://cutt.ly/syllaby