Writing DICE Tests¶
Anatomy of a DICE Project¶
The file structure of a basic DICE project likes:
project_root
|-- oracles
| `-- pyramid.yaml
`-- utils
`-- item.py
- An
item.py
is a python script defines how a single test item is run and pass the result to DICE for analysis. - The
oracles
directory contains one or more YAML files defines the expected results for different conditions. - The optional
utils
directory contains python helper modules to assist specific tests.
Writing Test Runner¶
item.py
contains a class Item
inherits from dice.item
class from DICE`s
core API:
import os
from dice.core import item
from dice import utils
class Item(item.ItemBase):
def run(self):
cmdline = os.path.join(self.provider.path, 'pyramid')
cmdline += ' %s' % utils.escape(str(self.get('option')))
self.res = utils.run(cmdline)
Writing Oracle¶
An example oracle YAML file likes:
- name: option
oracle: |
if option is Integer:
if option < 0:
return FAIL('Min input is 0')
elif option > 9223372036854775808:
return FAIL('Number overflow')
elif option > 1000:
return FAIL('Max input is 1000')
else:
return SUCCESS()
else:
return FAIL('Invalid number')
Every oracle YAML file contains a list of oracle objects. For each oracle, there is some predefined properties.
name
is the identifier of a specific oracle. It is recommended in CamelCase style to be differentiated from other variables.target
is where this oracle is applied to for the test item.tree
is a python style code snippet shows the expected result for a given conditions.