Labs:Perspective Stability

From AI Dungeon Wiki
Revision as of 00:46, 9 August 2020 by RockstarRaccoon (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

These tests will involve taking all three perspectives and seeing how well the AI functions with them.

Experiment Design

"Stability" is a general term here, and will be rated in terms of "failures".

Run some actions with the AI using 3 different perspectives on contexts which are otherwise identical, to see what the rate of "failures" are with each. These will be decided by a group of 3 and be totaled over a series of tests.

The following are usually annoying but in-bounds for the AI, able to be continued on with alteration, and will be considered "minor failures"...

  • Responses which are too short to be meaningful.
  • Responses which would not work in context, but still acknowledge it.
  • Including characters who are not there.
  • Changing minor details.
  • Taking the story off on a tangent.
  • Switching perspectives ("You" in third or first person, the character's name in first or second person, etc)
  • Attempting to create a scene-break but otherwise keeping sane.

The following are outright failures of the AI to continue the story, and are almost completely unusable, and will be considered "major failures"...

  • Responses which seem completely unrelated to given context.
  • Literal Nonsense

These tests can (and should) use a diverse range of contexts, including various prompts, but should also make an attempt to both fill both history and memory and to avoid using scenarios which reference a perspective with WorldInfo. Each context should have a significant number of attempts made within it using all 3 perspectives.

There should also be a secondary test run with > commands.

The Tests

Tests to be run after review of the above criteria.