FEL Testing: Icarus Verilog

This is a living blog page which entails up-to-date metrics.

This is a test plan for the QA of  Icarus Verilog used by Free Electronic Lab team before releasing it to the mirrors.

For every FEL iverilog release, the packager maintainer of iverilog is supposed to sync the contents of the ”’Expected”’ section with respect to the overcomes of the latest git version of ivtest.

Tests

ivtests suite

Icarus Verilog developers prepare a regression testsuite called ”ivtest”. There are three tests :

• vvp tests
• vpi tests
• vhdl tests

Details of the last test conducted :

• release : iverilog-0.9.20111101-1.f15.i686
• tester (FAS username) : chitlesh
• date : 01 November 2011

Tkdiff is a very robust and lightweight diff tools used by many CAD engineers in many respectable IC design centers. The FEL team uses tkdiff too to verify the tests against golden results.

Requirements

# yum install git tkcvs ghdl
$ git clone git://github.com/steveicarus/ivtest.git

If testing is being carried out before iverilog official release, ensure that you are using the latest iverilog version pushed to ”updates-testing” repository.

# yum install iverilog --enablerepo=updates-testing

VVP tests

Execution

$ cd ivtest
$ perl vvp_reg.pl

It should run for a coupled of minutes.

Test Results

Version 0.9.5

Test results:
Total=1704, Passed=1539, Failed=1, Not Implemented=129, Expected Fail=85
pr1877743: ==> Failed - output does not match gold file.

Version 0.9.4

Test results:
Total=1654, Passed=1522, Failed=8, Not Implemented=90, Expected Fail=34

Version 0.9.3

Total=1542, Passed=1505, Failed=1, Not Implemented=10, Expected Fail=26
pr1877743: ==> Failed - output does not match gold file.

Version 0.9.2

Total=1456, Passed=1440, Failed=1, Not Implemented=6, Expected Fail=9
pr1877743: ==> Failed - output does not match gold file.

This has a bug report in the tracker and should be fixed some time next year when a new expression type for the run time is created.

Verify against golden

$ tkdiff regression_report-v0.9.txt regression_report.txt &

VPI tests

Execution

$ cd ivtest
$ perl vpi_reg.pl

It should run for a coupled of seconds.

Test Results

Version 0.9.5

Test results: Total=39, Passed=36, Failed=0, Not Implemented=3

Version 0.9.4

Test results: Total=38, Passed=35, Failed=1, Not Implemented=2

Version 0.9.3

Total=38, Passed=36, Failed=0, Not Implemented=2

Version 0.9.2

Total=35, Passed=34, Failed=0, Not Implemented=1

VHDL tests

Execution

$ cd ivtest
$ ./vhdl_reg.pl vhdl_regress.list

It should run for a coupled of minutes.

Test Results

Version 0.9.5

Test results:
  Total=294, Passed=278, Failed=16, Not Implemented=0, Expected Fail=0

Version 0.9.4

Test results:
  Total=293, Passed=278, Failed=15, Not Implemented=0, Expected Fail=0

Version 0.9.3

Total=289, Passed=283, Failed=6, Not Implemented=0, Expected Fail=0
reserved: ==> Failed - running ghdl.
signed5: ==> Failed - output does not match gold file.
tri0: ==> Failed - output does not match gold file.
tri0b: ==> Failed - output does not match gold file.
tri1: ==> Failed - output does not match gold file.
wireland: ==> Failed - running ghdl.

Version 0.9.2

Total=287, Passed=283, Failed=4, Not Implemented=0, Expected Fail=0
tri0: ==> Failed - output does not match gold file.
tri0b: ==> Failed - output does not match gold file.
tri1: ==> Failed - output does not match gold file.
wireland: ==> Failed - running ghdl.

In the process of testing the verilog to vhdl translation, the following bugs were encountered:

• Log files can be found here.
• GHDL Bug filed: sr #2463: internal error: process in timeout – ”Bug report closed”
• Test report to iverilog upstream.

Verify against golden

$ tkdiff vhdl_regression_report-devel.txt vhdl_regression_report.txt &

=== Compiled extra testsuite ===

Some extra test cases have been compiled. They are located on FEL’s git repository.

{{{

#!sh

$ git clone ssh://git.fedorahosted.org/git/fedora-electronic-lab.git

$ cd fedora-electronic-lab

$ git checkout remotes/origin/testsuite

$ cd iverilog

}}}

==== Big and Little endian vectors ====

”Specification” : The file select.v and select2.v exercise vector select on both big and little endian vectors.

{{{

#!sh

$ cd endian

}}}

”Expected behaviour” : Each select line to produce the same value.

”Test 01”

{{{

#!sh

$ iverilog select.v -o select

$ ./select

}}}

”Outcomes of Test 01”

”Test 02”

{{{

#!sh

$ iverilog select2.v -o select2

$ ./select2

}}}

”Outcomes of Test 02”

”Notes:”

The relevant snippet from 1364-2005 (page 56-57):

{{{

reg [15:0] big_vect;

reg [0:15] little_vect;

big_vect[lsb_base_expr +: width_expr]

little_vect[msb_base_expr +: width_expr]

big_vect[msb_base_expr -: width_expr]

little_vect[lsb_base_expr -: width_expr]

}}}

The first two examples select bits starting at the base and ascending the bit range. The number of bits selected is equal to the width

expression. The second two examples select bits starting at the base and descending the bit range.

{{{

For example:

reg [31: 0] big_vect;

reg [0 :31] little_vect;

reg [63: 0] dword;

integer sel;

big_vect[ 0 +: 8]    // == big_vect[ 7 : 0]

big_vect[15 -: 8]    // == big_vect[15 : 8]

little_vect[ 0 +: 8] // == little_vect[0 : 7]

little_vect[15 -: 8] // == little_vect[8 :15]

dword[8*sel +: 8]    // variable part-select with fixed width

}}}

So it seems the + and – work on the numerical value, not the bit position. So big[0+:4] gives big![3:0] and little[0+:4] gives little![0:3].  Likewise, little[3+:4] gives little![3:6] or 1001 = 9 in the code, not little![0:3].

==== Testbench first ====

”Specification” :Can iverilog compile the design with *.v as arguments, if timescale is used ?

{{{

#!sh

$ cd t_TB_first

}}}

”Expected behaviour” :

* Test 01 fails.

* Test 02 succeeds.

”Test 01”

{{{

#!sh

$ iverilog *.v

}}}

”Outcomes of Test 01”

It should print an error message when the user tries to create a greater than 32 bit delay value in a continuous assignment. This will

* prevent the assert,

* point to the exact line and along with the timescale warning

* should give the user enough to debug the problem.

It is very unlikely that the user will be creating a real immediate delay that is more than 32 bits, so adding this may only happen in development.

It should ”’NOT”’ print the following (If it is, then update to the latest version of iverilog which Fedora provides)

{{{

ivl: eval_expr.c:116: get_number_immediate: Assertion `idx < 32′ failed.

sh: line 1: 12212 Done                    /usr/lib/ivl/ivlpp -L -F/tmp/ivrlg2262cbea1 -f/tmp/ivrlg262cbea1 -p/tmp/ivrli262cbea1

12213 Aborted                 (core dumped) | /usr/lib/ivl/ivl -C/tmp/ivrlh262cbea1 -C/usr/lib/ivl/vvp.conf — -

}}}

”Test 02”

{{{

#!sh

$ iverilog TestBench.v AndOr.v Intro_Top.v SR.v XorNor.v

}}}

”Outcomes of Test 02”

iverilog compiles successfully.