ANA в 11:23, 23 октября 2012

2012-10-23T11:23:27Z

Новая страница

{{OS-VVM TOC}}

==Open Source - VHDL Verification Methodology==

'''''Goal:'''''
* Learn how to add functional coverage, constrained random, and coverage driven random to your current VHDL testbench.

'''''Topics'''''
* Introduction
* Methodology
* What is Functional Coverage?
* Code coverage is not enough
* Test Done = Test Plan Executed
* Capturing Point/Item Coverage
* Capturing Cross Coverage
* Intelligent Coverage
* Refinement of Intelligent Coverage
* Randomization

== OS-VVM Introduction ==

* New features based on packages rather than syntax

* CoveragePkg
** Simplifies modeling high fidelity functional coverage
** Provides coverage driven randomization = intelligent coverage

* RandomPkg (supported by RandomBasePkg and SortListPkg_int)
** Simplifies randomization
** Provides methodology to implement constrained random

* Advantage of a package based approach
** Available now - just turn on VHDL-2008
** Works in your current testbench
** Open source packages will be updated as needed

== OS-VVM Methodology==
* Goal of Verification: Achieve Complete Coverage of the Test Plan

* OS-VVM process
** Add Functional Coverage
*** Purpose: Observes the design and indicates when testing is done
** Randomize using the coverage model = intelligent coverage
*** Purpose: Balances the randomization (makes solver unnecessary)
*** Simplifies remaining randomization to a refinement step
** Refine the coverage based randomization with sequential code
*** Purpose: Use directed, algorithmic, file-based, and constrained random methods to further refine the stimulus generation.
*** Can use multiple transactions to reach a coverage goal

*OS-VVM Advantages
** Designed to integrate with your current testbench
** Use methodology in part or in whole.

== What is Functional Coverage? ==

* Code that measures execution of test plan
** Tracks requirements, features, and boundary conditions

* Point / Item Coverage
** Relationships within a single object
** Bins of values, such as transfer sizes:
*** 1, 2, 3, 4-127, 128-252, 253, 254, 255

* Cross Coverage
** Relationships between multiple objects
** Has the each pair of registers been used with the ALU?

* When to Collect Coverage
** At an event (rising_edge(Clk))
** When a transaction completes

* Functional Coverage @ 100 % = Test Plan Executed

== Code Coverage is not enough ==

* Code coverage is automatically collected by the simulator
** Line coverage: Executed line of code
** Expression coverage: True / False analysis on expressions
** Statemachine Coverage: Track State and Arc execution

* Code coverage cannot analyze items that are not directly in the code
** Bins of values: 1, 2, 3, 4-127, 128-252, 253, 254, 255
** Pairs of registers been used with an ALU
*** Is each pair selected in a line of code? Usually they are separate.

* Code coverage will find not items missing from the design (not coded)
* Code coverage is optimistic since it runs when the process runs
** For combinational logic this is delta cycles

<source lang="vhdl">PrioritySel : process (SelA, SelB, SelC, A, B, C)
...
</source>

* Code Coverage @ 100 % /= done

== Test Done <nowiki>=</nowiki> Test Plan Executed ==

* Use functional coverage to capture your test plan
** With high fidelity functional coverage @ 100% = Test Done

* FC @ 100 % and Code Coverage < 100 % = ?
** Not done
** Untested code which is not in the test plan
** Either test plan is not complete or design contains extra features = bad!

* Use code coverage as a fail safe for the functional coverage

== Why You Need Functional Coverage ==

* "I have written a directed test for each item in the test plan, I am done right?"
** For a small design maybe

* As complexity grows and the design evolves, are you sure?
** When the FIFO size quadruples, does the test still fill it?
** Add bits/features to a configuration register, do all tests set it correctly?

* To avoid missing items, use functional coverage for all stimulus generation.
** Rather than assume, functional coverage observes that the test plan points actually get exercised.

==Point / Item Coverage==

* Relationships within a single object = Bins of values.

Transfer Sizes Count
1
2
3
4 to 127
128 to 252
253
254
255

* For data transfers, typically boundary conditions occur at smaller and larger transfer sizes and the middle transfers are of less interest
* Methods:
** Manual
** Using CoveragePkg

== Point / Item Coverage: Manual ==

<source lang="vhdl">
signal Bin : integer_vector(1 to 8) ; -- Declaration of 8 Bins
. . .
process
begin
wait until rising_edge(Clk) and DValid = '1' ; -- Sampling
case to_integer(ActualData) is
when 1 => Bin(1) <= Bin(1) + 1 ; -- Bin Creation
when 2 => increment( Bin(2) ) ; --
when 3 => increment( Bin(3) ) ; --
when 4 to 127 => increment( Bin(4) ) ; --
when 128 to 252 => increment( Bin(5) ) ; --
when 253 => increment( Bin(6) ) ; --
when 254 => increment( Bin(7) ) ; --
when 255 => increment( Bin(8) ) ; --
when others => null ;
end case ;
end process ;
</source>

{{Info|'''''Issues: Too much work, Too specific to a problem, No reuse, No built-in reporting'''''}}

== CoveragePkg ==

* Models functional coverage using methods & variables in a protected type

:{|
|<source lang="vhdl">
function GenBin ( . . . ) return CovBinType ;

type CovPType is protected
procedure AddBins ( CovBin : CovBinType ) ;
procedure AddCross( Bin1, Bin2, ... : CovBinType ) ;
procedure ICover ( val : integer ) ;
procedure ICover ( val : integer_vector ) ;
impure function IsCovered ( ... ) return boolean ;
procedure WriteBin ;
procedure WriteCovHoles( PercentCov : Real := 100.0) ;
. . .
end protected CovPType ;
</source>
|}

* Coverage data structure constructed in a shared variable

:<source lang="vhdl">shared variable Bin1 : CovPType ;</source>

* Calling protected type methods requires referencing the variable

:<source lang="vhdl">Bin1.AddBins ( GenBin(1, 3, 3) ) ;</source>

==Point / Item Coverage: CoveragePkg==

<source lang="vhdl">
architecture Test1 of tb is
shared variable Bin1 : CovPType ; -- Cov object (Bin1) contains
-- data struct and cfg
begin
CollectCov : process
begin
-- AddBins builds data structure
-- GenBin defines bin values
Bin1.AddBins( GenBin( 1, 3, 3 )) ; -- Create Bins
Bin1.AddBins( GenBin( 4, 252, 2 )) ; -- Create Bins
Bin1.AddBins( GenBin( 253, 255 )) ; -- Create Bins
loop
wait until rising_edge(Clk) and DValid = '1';
Bin1.ICover(to_integer(ActualData)); -- Collect Cov
end loop ;
end process ;

ReportCov : process
begin
-- ReportCov
wait until rising_edge(Clk) and Bin1.IsCovered ;
Bin1.WriteBin ;
end process ;
</source>

== Bin Construction: AddBins + GenBin==

* Method AddBins: Creates internal data structure in CovPType.

* GenBin: Creates an array of bins, the input to AddBins

<source lang="vhdl">-- min, max, #bins
CovBin1.AddBins( GenBin( 1, 3, 3 ));</source>

:* Create 3 bins with ranges: 1 to 1, 2 to 2, and 3 to 3 .

* Additional calls to AddBins creates additional bins
CovBin1.AddBins( GenBin( 4, 252, 2 ) );
:*Creates 2 additional bins with ranges: 4 to 127, 128 to 252.

* GenBin without NumBins creates one bin per value
CovBin1.AddBins( GenBin( 253, 255 ) );
:* 3 additional bins with ranges: 253 to 253, 254 to 254, and 255 to 255.

* GenBin a single parameter creates one bin: 1 to 1
CovBin1.AddBins( GenBin( 1 ) );

==Cross Coverage==

* Testing an ALU with Multiple Inputs:

[[File:OS-VVM-Pic-CrossCoverage-Example.png|center]]

* Need to test every register in SRC1 with every register in SRC2

{| class=standard align=center
! colspan=2 rowspan=2 | || colspan=8| SRC2
|-
! R0 || R1 || R2 || R3 || R4 || R5 || R6 || R7
|-
! rowspan=8| SRC1 || R0 || || || || || || || ||
|-
! R1 || || || || || || || ||
|-
! R2 || || || || || || || ||
|-
! R3 || || || || || || || ||
|-
! R4 || || || || || || || ||
|-
! R5 || || || || || || || ||
|-
! R6 || || || || || || || ||
|-
! R7 || || || || || || || ||
|}

* Result: Matrix of conditions that must be covered

== Cross Coverage==

<source lang="vhdl">
architecture Test3 of tb is
shared variable ACov : CovPType ;
-- Cov Object
begin
CollectCov : process
variable RV : RandomPType ; -- randomization object
begin
Create Bins
ACov.AddCross( GenBin(0,7), GenBin(0,7) );
8x8 Matrix
while not Done loop
RegIn1 := RV.RandInt(0, 7) ;
Randomize
RegIn2 := RV.RandInt(0, 7) ;
Reg Addr
DoAluOp(TRec, RegIn1, RegIn2) ;
Do Transaction
ACov.ICover( (RegIn1, RegIn2) ) ;
Collect Cov
end loop ;
ACov.WriteBin ;
-- Report Coverage
EndStatus(. . . ) ;
end process ;
Matrix coverage creates many bins
in a quick simple format.
</source>

:{|
|<source lang="vhdl">
architecture Test2 of tb is
shared variable ACov : CovPType ; -- Declare Cov Object
begin
TestProc : process
variable RV : RandomPType ;
variable RegIn1, RegIn2 : integer ;
begin
ACov.AddCross( GenBin(0,7), GenBin(0,7) ); -- Model
while not ACov.IsCovered loop -- Interact
-- Randomize register addresses -- see RandomPkg documentation
RegIn1 := RV.RandInt(0, 7) ;
RegIn2 := RV.RandInt(0, 7) ;
DoAluOp(TRec, RegIn1, RegIn2) ; -- Do a transaction
ACov.ICover( (RegIn1, RegIn2) ) ; -- Accumulate
end loop ;
ACov.WriteBin ; -- Report
EndStatus(. . . ) ;
end process ;
</source>
|}

== Problem with Constrained Random==
* Problem:
** CR repeats some test cases before generating all conditions
** In general, it takes N*log N randomizations to cover N conditions
** The "log N" factor can add significantly to simulation run times.

* Running the previous ALU test takes 315 > 64 * Log 64 iterations
** 315 is approximately 5X too many iterations
** Using a different seed can increase or decrease this

{| class=standard align=center
! colspan=2 rowspan=2 | || colspan=8| SRC2
|-
! R0 || R1 || R2 || R3 || R4 || R5 || R6 || R7
|-
! rowspan=8| SRC2 || R0
| 6 || 6 || 9 || {{Кр|'''1'''}} || 4 || 6 || 6 || 5
|-
! R1
| 3 || 4 || 3 || 6 || 9 || 5 || 5 || 4
|-
! R2
| 4 || {{Кр|'''1'''}} || 5 || 3 || 2 || 3 || 4 || 6
|-
! R3
| 5 || 5 || 6 || 3 || 3 || 4 || 4 || 6
|-
! R4
| 4 || 5 || 5 || 10 || 9 || 10 || 7 || 7
|-
! R5
| 4 || 6 || 3 || 6 || 3 || 5 || 3 || 8
|-
! R6
| 3 || 6 || 3 || 4 || 7 || {{Кр|'''1'''}} || 4 || 6
|-
! R7
| 7 || 3 || 4 || 6 || 6 || 5 || 4 || 5
|}

Open Source VHDL Verification Methodology/Презентация/en - История изменений

ANA в 11:23, 23 октября 2012