package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (41)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
sha256=8643b39e74317e2ec49be9a0986e73acd4fcd1277a6d59fb1423c3a6ebdfaf61
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. angstrom >= "0.7.0"
  8. ansi >= "0.6.0"
  9. anycache >= "0.7.4"
  10. anycache-async
  11. anycache-lwt
  12. archetype >= "1.4.2"
  13. archi
  14. arp
  15. arp-mirage
  16. arrakis
  17. art
  18. asak >= "0.2"
  19. asli >= "0.2.0"
  20. asn1-combinators >= "0.2.2"
  21. atd >= "2.3.3"
  22. atdgen >= "2.10.0"
  23. atdpy
  24. atdts
  25. base32
  26. base64 >= "2.1.2"
  27. bechamel >= "0.5.0"
  28. bigarray-overlap
  29. bigstring >= "0.3"
  30. bigstring-unix >= "0.3"
  31. bigstringaf
  32. bitlib
  33. blake2
  34. bloomf
  35. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  36. bls12-381-hash
  37. bls12-381-js >= "0.4.2"
  38. bls12-381-js-gen >= "0.4.2"
  39. bls12-381-legacy
  40. bls12-381-signature
  41. bls12-381-unix
  42. blurhash
  43. builder-web < "0.2.0"
  44. bulletml
  45. bytebuffer
  46. ca-certs
  47. ca-certs-nss
  48. cactus
  49. calendar >= "3.0.0"
  50. callipyge
  51. camlix
  52. camlkit
  53. camlkit-base
  54. capnp-rpc < "0.6.0"
  55. capnp-rpc-lwt < "0.3"
  56. carray
  57. carton
  58. cborl
  59. ccss >= "1.6"
  60. cf-lwt
  61. chacha
  62. channel
  63. charrua-client
  64. charrua-client-lwt
  65. charrua-client-mirage < "0.11.0"
  66. checkseum >= "0.0.3"
  67. cid
  68. clarity-lang
  69. class_group_vdf
  70. cohttp >= "0.17.0"
  71. cohttp-curl-async
  72. cohttp-curl-lwt
  73. cohttp-eio >= "6.0.0~beta2"
  74. colombe
  75. color
  76. conan
  77. conan-cli
  78. conan-database
  79. conan-lwt
  80. conan-unix
  81. conduit = "3.0.0"
  82. conex < "0.10.0"
  83. conex-mirage-crypto
  84. conex-nocrypto
  85. cookie
  86. cow >= "2.2.0"
  87. css
  88. css-parser
  89. cstruct >= "3.3.0"
  90. cstruct-sexp
  91. ctypes-zarith
  92. cuid
  93. curly
  94. current_incr
  95. cwe_checker
  96. data-encoding < "1.0.0"
  97. datakit >= "0.12.0"
  98. datakit-bridge-github >= "0.12.0"
  99. datakit-ci
  100. datakit-client-git >= "0.12.0"
  101. decompress >= "0.8" & < "1.5.3"
  102. depyt
  103. digestif >= "0.7"
  104. dispatch >= "0.4.1"
  105. dkim
  106. dkim-bin
  107. dkim-mirage
  108. dns >= "4.0.0"
  109. dns-cli
  110. dns-client >= "4.6.0"
  111. dns-forward < "0.9.0"
  112. dns-forward-lwt-unix
  113. dns-resolver
  114. dns-server
  115. dns-tsig
  116. dnssd
  117. dnssec
  118. docfd >= "2.2.0"
  119. dog < "0.2.1"
  120. domain-name
  121. dot-merlin-reader >= "5.3~5.3preview"
  122. dream
  123. dream-pure
  124. duff
  125. dune-release >= "1.0.0"
  126. duration >= "0.1.1"
  127. emile
  128. encore
  129. eqaf >= "0.5"
  130. equinoxe
  131. equinoxe-cohttp
  132. equinoxe-hlc
  133. eris
  134. eris-lwt
  135. ezgzip
  136. ezjsonm >= "0.4.2" & < "1.3.0"
  137. ezjsonm-lwt < "1.3.0"
  138. FPauth
  139. FPauth-core
  140. FPauth-responses
  141. FPauth-strategies
  142. faraday != "0.2.0"
  143. farfadet
  144. fat-filesystem >= "0.12.0"
  145. ff
  146. ff-pbt
  147. fiat-p256
  148. flex-array
  149. fsevents-lwt
  150. functoria >= "2.2.0"
  151. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  152. geojson
  153. geoml >= "0.1.1"
  154. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  155. git-mirage < "3.0.0"
  156. git-split
  157. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  158. git_split
  159. gitlab-unix
  160. glicko2
  161. gmap >= "0.3.0"
  162. gpt
  163. graphql
  164. graphql-async
  165. graphql-cohttp >= "0.13.0"
  166. graphql-lwt
  167. graphql_parser
  168. graphql_ppx >= "0.7.1"
  169. h1_parser
  170. h2
  171. hacl
  172. hacl_func
  173. hacl_x25519 >= "0.2.0"
  174. highlexer
  175. hkdf
  176. hockmd
  177. html_of_jsx
  178. http
  179. http-multipart-formdata < "2.0.0"
  180. httpaf >= "0.2.0"
  181. httpun
  182. httpun-ws
  183. hvsock
  184. icalendar
  185. imagelib >= "20200929"
  186. index
  187. inferno >= "20220603"
  188. influxdb-async
  189. influxdb-lwt
  190. inquire < "0.2.0"
  191. interval-map
  192. iomux
  193. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  194. irmin-bench >= "2.7.0"
  195. irmin-chunk < "2.0.0" | >= "2.3.0"
  196. irmin-cli
  197. irmin-containers
  198. irmin-fs < "2.0.0" | >= "2.3.0"
  199. irmin-git < "2.0.0" | >= "2.3.0"
  200. irmin-http < "2.0.0"
  201. irmin-mem < "2.0.0"
  202. irmin-pack >= "2.4.0" & != "2.6.1"
  203. irmin-pack-tools
  204. irmin-test < "2.2.0"
  205. irmin-tezos
  206. irmin-tezos-utils
  207. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  208. irmin-watcher
  209. jekyll-format
  210. jerboa
  211. jitsu
  212. jose
  213. json-data-encoding >= "0.9"
  214. json_decoder
  215. jsonxt
  216. junit_alcotest
  217. jwto
  218. kdf
  219. ke >= "0.2"
  220. kkmarkdown
  221. lambda-runtime
  222. lambdapi >= "2.0.0"
  223. lambdoc >= "1.0-beta4"
  224. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  225. lmdb >= "1.0"
  226. logical
  227. logtk >= "1.5.1"
  228. lp
  229. lp-glpk
  230. lp-glpk-js
  231. lp-gurobi
  232. lru
  233. lt-code
  234. luv
  235. mbr-format >= "1.0.0"
  236. mdx >= "1.6.0"
  237. mec
  238. mechaml >= "1.0.0"
  239. merge-queues >= "0.2.0"
  240. merge-ropes >= "0.2.0"
  241. merlin >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  242. merlin-lib >= "4.17.1-414" & < "5.0-502" | >= "5.2.1-502"
  243. metrics
  244. mirage >= "4.0.0~beta1"
  245. mirage-block-partition < "0.2.0"
  246. mirage-block-ramdisk >= "0.3"
  247. mirage-channel >= "4.0.0"
  248. mirage-channel-lwt
  249. mirage-crypto-ec
  250. mirage-flow >= "1.0.2" & < "1.2.0"
  251. mirage-flow-unix
  252. mirage-fs-mem
  253. mirage-fs-unix >= "1.2.0"
  254. mirage-kv >= "2.0.0"
  255. mirage-kv-mem
  256. mirage-kv-unix
  257. mirage-logs >= "0.3.0"
  258. mirage-nat
  259. mirage-net-unix >= "2.3.0"
  260. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  261. mirage-tc
  262. mjson
  263. mnd
  264. monocypher
  265. mrmime
  266. mrt-format
  267. multibase
  268. multihash
  269. multihash-digestif
  270. multipart-form-data
  271. multipart_form
  272. multipart_form-eio
  273. multipart_form-lwt
  274. named-pipe
  275. nanoid
  276. nbd >= "4.0.3"
  277. nbd-tool
  278. nloge
  279. nocoiner
  280. non_empty_list
  281. OCADml >= "0.6.0"
  282. obatcher
  283. ocaml-index >= "1.1"
  284. ocaml-r >= "0.5.0"
  285. ocaml-version >= "3.1.0"
  286. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  287. ocamlformat-rpc < "removed"
  288. ocamline
  289. odoc >= "1.4.0" & < "2.1.0"
  290. ohex
  291. oidc
  292. opam-0install
  293. opam-0install-cudf >= "0.5.0"
  294. opam-file-format >= "2.1.1"
  295. opentelemetry >= "0.6"
  296. opentelemetry-client-cohttp-lwt >= "0.6"
  297. opentelemetry-client-ocurl >= "0.6"
  298. opentelemetry-cohttp-lwt >= "0.6"
  299. opentelemetry-lwt >= "0.6"
  300. opium >= "0.15.0"
  301. opium-graphql
  302. opium-testing
  303. opium_kernel
  304. orewa
  305. ortac-core
  306. osx-acl
  307. osx-attr
  308. osx-cf
  309. osx-fsevents
  310. osx-membership
  311. osx-mount
  312. osx-xattr
  313. otoggl
  314. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  315. owl-base < "0.5.0"
  316. owl-ode >= "0.1.0" & != "0.2.0"
  317. owl-symbolic
  318. passmaker
  319. patch
  320. pbkdf
  321. pecu >= "0.2"
  322. pf-qubes
  323. pg_query >= "0.9.6"
  324. phylogenetics
  325. piaf
  326. polyglot
  327. polynomial
  328. ppx_blob >= "0.3.0"
  329. ppx_deriving_cmdliner
  330. ppx_deriving_rpc
  331. ppx_deriving_yaml
  332. ppx_graphql >= "0.2.0"
  333. ppx_mysql
  334. ppx_protocol_conv >= "5.0.0"
  335. ppx_protocol_conv_json >= "5.0.0"
  336. ppx_protocol_conv_jsonm >= "5.0.0"
  337. ppx_protocol_conv_msgpack >= "5.0.0"
  338. ppx_protocol_conv_xml_light >= "5.0.0"
  339. ppx_protocol_conv_xmlm
  340. ppx_protocol_conv_yaml >= "5.0.0"
  341. ppx_subliner
  342. ppx_units
  343. ppx_yojson >= "1.1.0"
  344. pratter
  345. prc
  346. preface
  347. pretty_expressive
  348. prettym
  349. proc-smaps
  350. producer < "0.2.0"
  351. prometheus < "1.2"
  352. prometheus-app
  353. protocell
  354. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  355. protocol-9p-unix
  356. psq
  357. qcheck >= "0.18" & < "0.22"
  358. qcheck-alcotest < "0.22"
  359. quickjs
  360. radis
  361. randii
  362. reason-standard
  363. red-black-tree
  364. reparse >= "2.0.0" & < "3.0.0"
  365. reparse-unix < "2.1.0"
  366. resp < "0.10.0"
  367. resp-unix
  368. rfc1951 < "1.0.0"
  369. routes < "2.0.0"
  370. rpc >= "5.9.0"
  371. rpclib
  372. rpclib-async
  373. rpclib-lwt
  374. rpmfile < "0.3.0"
  375. rpmfile-eio
  376. rpmfile-unix
  377. rubytt
  378. SZXX >= "4.0.0"
  379. salsa20
  380. salsa20-core
  381. sanddb >= "0.2"
  382. scaml >= "1.5.0"
  383. scrypt-kdf
  384. secp256k1 >= "0.4.1"
  385. secp256k1-internal
  386. semver >= "0.2.1"
  387. sendmail
  388. sendmail-lwt
  389. sendmail-miou-unix
  390. sendmail-mirage
  391. sendmsg
  392. server-reason-react
  393. session-cookie
  394. session-cookie-async
  395. session-cookie-lwt
  396. sherlodoc
  397. slug
  398. sodium-fmt
  399. spin >= "0.6.0"
  400. squirrel
  401. ssh-agent
  402. ssl >= "0.6.0"
  403. stramon-lib
  404. styled-ppx
  405. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  406. tdigest < "2.1.0"
  407. term-indexing
  408. term-tools
  409. terminal_size >= "0.1.1"
  410. terminus
  411. terminus-cohttp
  412. terminus-hlc
  413. terml
  414. testo
  415. testo-lwt
  416. textrazor
  417. tezos-base-test-helpers < "13.0"
  418. tezos-client-base < "12.0"
  419. tezos-lmdb
  420. tezos-test-helpers < "11.0"
  421. tftp
  422. timedesc
  423. timere
  424. tls >= "0.12.0"
  425. toc
  426. topojson
  427. topojsone
  428. transept
  429. twostep
  430. type_eq
  431. type_id
  432. typebeat
  433. typeid >= "1.0.1"
  434. tyre >= "0.4"
  435. tyxml >= "4.0.0"
  436. tyxml-jsx
  437. tyxml-ppx >= "4.3.0"
  438. tyxml-syntax
  439. uecc
  440. ulid
  441. universal-portal
  442. unix-dirent
  443. unix-errno >= "0.3.0"
  444. unix-fcntl >= "0.3.0"
  445. unix-sys-resource
  446. unix-sys-stat
  447. unix-time
  448. unstrctrd
  449. user-agent-parser
  450. uspf
  451. uspf-lwt
  452. uspf-mirage
  453. uspf-unix
  454. utop >= "2.13.0"
  455. validate
  456. validator
  457. vercel
  458. vpnkit
  459. wcwidth
  460. websocketaf
  461. x509 >= "0.7.0"
  462. xapi-rrd >= "1.8.2"
  463. xapi-stdext-date
  464. xapi-stdext-encodings
  465. xapi-stdext-std >= "4.16.0"
  466. yaml < "3.2.0"
  467. yaml-sexp
  468. yocaml < "2.0.0"
  469. yocaml_syndication = "2.0.0"
  470. yocaml_yaml < "2.0.0"
  471. yuscii >= "0.2.0"
  472. zar
  473. zed >= "3.2.2"
  474. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct