Labs:Perspective Stability

From AI Dungeon Wiki
Jump to navigation Jump to search

These tests will involve taking all three perspectives and seeing how well the AI functions with them.

Experiment Design

"Stability" is a general term here, and will be rated in terms of "failures".

Run some actions with the AI using 3 different perspectives on contexts which are otherwise identical, to see what the rate of "failures" are with each. These will be decided by a group of 3 and be totaled over a series of tests.

The following are usually annoying but in-bounds for the AI, able to be continued on with alteration, and will be considered "minor failures"...

  • Responses which are too short to be meaningful.
  • Responses which would not work in context, but still acknowledge it.
  • Including characters who are not there.
  • Changing minor details.
  • Taking the story off on a tangent.
  • Switching perspectives ("You" in third or first person, the character's name in first or second person, etc)
  • Attempting to create a scene-break but otherwise keeping sane.

The following are outright failures of the AI to continue the story, and are almost completely unusable, and will be considered "major failures"...

  • Responses which seem completely unrelated to given context.
  • Literal Nonsense

These tests can (and should) use a diverse range of contexts, including various prompts, but should also make an attempt to both fill both history and memory and to avoid using scenarios which reference a perspective with WorldInfo. Each context should have a significant number of attempts made within it using all 3 perspectives.

There should also be a secondary test run with > commands.

The Tests

Tests to be run after review of the above criteria.