OTBN Simulation Results

Sunday March 23 2025 00:10:17 UTC

GitHub Revision: `27fc640f8d`

Branch: master

Testplan

Simulator: XCELIUM

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	otbn_smoke	19.000s	124.965us	1	1	100.00
V1	single_binary	otbn_single	2.050m	483.243us	100	100	100.00
V1	csr_hw_reset	otbn_csr_hw_reset	9.000s	18.412us	5	5	100.00
V1	csr_rw	otbn_csr_rw	12.000s	20.784us	20	20	100.00
V1	csr_bit_bash	otbn_csr_bit_bash	11.000s	291.178us	5	5	100.00
V1	csr_aliasing	otbn_csr_aliasing	8.000s	44.817us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	otbn_csr_mem_rw_with_rand_reset	18.000s	48.799us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	otbn_csr_rw	12.000s	20.784us	20	20	100.00
		otbn_csr_aliasing	8.000s	44.817us	5	5	100.00
V1	mem_walk	otbn_mem_walk	31.000s	2.190ms	5	5	100.00
V1	mem_partial_access	otbn_mem_partial_access	26.000s	2.456ms	5	5	100.00
V1		TOTAL			166	166	100.00
V2	reset_recovery	otbn_reset	1.283m	367.953us	9	10	90.00
V2	multi_error	otbn_multi_err	1.833m	415.357us	1	1	100.00
V2	back_to_back	otbn_multi	5.033m	1.109ms	10	10	100.00
V2	stress_all	otbn_stress_all	1.483m	325.714us	10	10	100.00
V2	lc_escalation	otbn_escalate	34.000s	67.292us	60	60	100.00
V2	zero_state_err_urnd	otbn_zero_state_err_urnd	11.000s	18.093us	5	5	100.00
V2	sw_errs_fatal_chk	otbn_sw_errs_fatal_chk	30.000s	732.271us	10	10	100.00
V2	alert_test	otbn_alert_test	12.000s	30.917us	50	50	100.00
V2	intr_test	otbn_intr_test	13.000s	28.249us	50	50	100.00
V2	tl_d_oob_addr_access	otbn_tl_errors	15.000s	90.085us	20	20	100.00
V2	tl_d_illegal_access	otbn_tl_errors	15.000s	90.085us	20	20	100.00
V2	tl_d_outstanding_access	otbn_csr_hw_reset	9.000s	18.412us	5	5	100.00
		otbn_csr_rw	12.000s	20.784us	20	20	100.00
		otbn_csr_aliasing	8.000s	44.817us	5	5	100.00
		otbn_same_csr_outstanding	14.000s	30.015us	20	20	100.00
V2	tl_d_partial_access	otbn_csr_hw_reset	9.000s	18.412us	5	5	100.00
		otbn_csr_rw	12.000s	20.784us	20	20	100.00
		otbn_csr_aliasing	8.000s	44.817us	5	5	100.00
		otbn_same_csr_outstanding	14.000s	30.015us	20	20	100.00
V2		TOTAL			245	246	99.59
V2S	mem_integrity	otbn_imem_err	18.000s	60.992us	10	10	100.00
		otbn_dmem_err	15.000s	24.623us	15	15	100.00
V2S	internal_integrity	otbn_alu_bignum_mod_err	15.000s	188.025us	4	5	80.00
		otbn_controller_ispr_rdata_err	22.000s	165.562us	3	5	60.00
		otbn_mac_bignum_acc_err	13.000s	210.608us	5	5	100.00
		otbn_urnd_err	10.000s	14.556us	2	2	100.00
V2S	illegal_bus_access	otbn_illegal_mem_acc	10.000s	33.668us	5	5	100.00
V2S	otbn_mem_gnt_acc_err	otbn_mem_gnt_acc_err	9.000s	68.857us	2	2	100.00
V2S	otbn_non_sec_partial_wipe	otbn_partial_wipe	18.000s	71.685us	10	10	100.00
V2S	tl_intg_err	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
		otbn_tl_intg_err	36.000s	129.912us	20	20	100.00
V2S	passthru_mem_tl_intg_err	otbn_passthru_mem_tl_intg_err	54.000s	292.203us	20	20	100.00
V2S	prim_fsm_check	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	prim_count_check	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_mem_scramble	otbn_smoke	19.000s	124.965us	1	1	100.00
V2S	sec_cm_data_mem_integrity	otbn_dmem_err	15.000s	24.623us	15	15	100.00
V2S	sec_cm_instruction_mem_integrity	otbn_imem_err	18.000s	60.992us	10	10	100.00
V2S	sec_cm_bus_integrity	otbn_tl_intg_err	36.000s	129.912us	20	20	100.00
V2S	sec_cm_controller_fsm_global_esc	otbn_escalate	34.000s	67.292us	60	60	100.00
V2S	sec_cm_controller_fsm_local_esc	otbn_imem_err	18.000s	60.992us	10	10	100.00
		otbn_dmem_err	15.000s	24.623us	15	15	100.00
		otbn_zero_state_err_urnd	11.000s	18.093us	5	5	100.00
		otbn_illegal_mem_acc	10.000s	33.668us	5	5	100.00
		otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_controller_fsm_sparse	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_scramble_key_sideload	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_scramble_ctrl_fsm_local_esc	otbn_imem_err	18.000s	60.992us	10	10	100.00
		otbn_dmem_err	15.000s	24.623us	15	15	100.00
		otbn_zero_state_err_urnd	11.000s	18.093us	5	5	100.00
		otbn_illegal_mem_acc	10.000s	33.668us	5	5	100.00
		otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_scramble_ctrl_fsm_sparse	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_start_stop_ctrl_fsm_global_esc	otbn_escalate	34.000s	67.292us	60	60	100.00
V2S	sec_cm_start_stop_ctrl_fsm_local_esc	otbn_imem_err	18.000s	60.992us	10	10	100.00
		otbn_dmem_err	15.000s	24.623us	15	15	100.00
		otbn_zero_state_err_urnd	11.000s	18.093us	5	5	100.00
		otbn_illegal_mem_acc	10.000s	33.668us	5	5	100.00
		otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_start_stop_ctrl_fsm_sparse	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_data_reg_sw_sca	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_ctrl_redun	otbn_ctrl_redun	16.000s	48.601us	12	12	100.00
V2S	sec_cm_pc_ctrl_flow_redun	otbn_pc_ctrl_flow_redun	11.000s	82.209us	5	5	100.00
V2S	sec_cm_rnd_bus_consistency	otbn_rnd_sec_cm	41.000s	140.893us	5	5	100.00
V2S	sec_cm_rnd_rng_digest	otbn_rnd_sec_cm	41.000s	140.893us	5	5	100.00
V2S	sec_cm_rf_base_data_reg_sw_integrity	otbn_rf_base_intg_err	21.000s	58.641us	10	10	100.00
V2S	sec_cm_rf_base_data_reg_sw_glitch_detect	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_stack_wr_ptr_ctr_redun	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_rf_bignum_data_reg_sw_integrity	otbn_rf_bignum_intg_err	1.333m	364.108us	9	10	90.00
V2S	sec_cm_rf_bignum_data_reg_sw_glitch_detect	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_loop_stack_ctr_redun	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S	sec_cm_loop_stack_addr_integrity	otbn_stack_addr_integ_chk	14.000s	36.598us	5	5	100.00
V2S	sec_cm_call_stack_addr_integrity	otbn_stack_addr_integ_chk	14.000s	36.598us	5	5	100.00
V2S	sec_cm_start_stop_ctrl_state_consistency	otbn_sec_wipe_err	23.000s	578.442us	2	7	28.57
V2S	sec_cm_data_mem_sec_wipe	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_instruction_mem_sec_wipe	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_data_reg_sw_sec_wipe	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_write_mem_integrity	otbn_multi	5.033m	1.109ms	10	10	100.00
V2S	sec_cm_ctrl_flow_count	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_ctrl_flow_sca	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_data_mem_sw_noaccess	otbn_sw_no_acc	18.000s	116.352us	5	5	100.00
V2S	sec_cm_key_sideload	otbn_single	2.050m	483.243us	100	100	100.00
V2S	sec_cm_tlul_fifo_ctr_redun	otbn_sec_cm	4.500m	1.236ms	2	5	40.00
V2S		TOTAL			151	163	92.64
V3	stress_all_with_rand_reset	otbn_stress_all_with_rand_reset	5.817m	2.651ms	2	10	20.00
V3		TOTAL			2	10	20.00
		TOTAL			564	585	96.41

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
98.97	99.57	94.94	99.66	93.32	93.59	94.87	90.79	99.57

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:891) [otbn_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 7 failures:
- Test otbn_stress_all_with_rand_reset has 7 failures.
  - 1.otbn_stress_all_with_rand_reset.96819925995805759911322431301232094218325421143843916554408676657065976226613
    Line 165, in log /nightly/runs/scratch/master/otbn-sim-xcelium/1.otbn_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 859490010 ps: (cip_base_vseq.sv:891) [uvm_test_top.env.virtual_sequencer.otbn_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 859490010 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
  - 3.otbn_stress_all_with_rand_reset.43683113196722297234030598961871078054290155854639256347088638163443777833856
    Line 173, in log /nightly/runs/scratch/master/otbn-sim-xcelium/3.otbn_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 204211653 ps: (cip_base_vseq.sv:891) [uvm_test_top.env.virtual_sequencer.otbn_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 204211653 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
  - ... and 5 more failures.

xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_dv_otbn_env_*/otbn_insn_cnt_if.sv,21): Assertion InsnCntMatches_A has failed has 5 failures:

Test otbn_sec_wipe_err has 5 failures.

0.otbn_sec_wipe_err.52098559212843507990100249512991397520819704114030772930671927081561754640139
Line 120, in log /nightly/runs/scratch/master/otbn-sim-xcelium/0.otbn_sec_wipe_err/latest/run.log

  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 26550048 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @  26550048 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @  26550048 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

1.otbn_sec_wipe_err.110935402178777550740389534345514248355788144311727561467380832761654725611515
Line 112, in log /nightly/runs/scratch/master/otbn-sim-xcelium/1.otbn_sec_wipe_err/latest/run.log

  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 30389610 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @  30389610 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @  30389610 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

UVM_ERROR (cip_base_scoreboard.sv:249) scoreboard [scoreboard] alert fatal has unexpected timeout error has 4 failures:
- Test otbn_rf_bignum_intg_err has 1 failures.
  - 0.otbn_rf_bignum_intg_err.25033754076814127799158489805386426413482297015206790806604126175784918828396
    Line 116, in log /nightly/runs/scratch/master/otbn-sim-xcelium/0.otbn_rf_bignum_intg_err/latest/run.log
```
  UVM_ERROR @ 310272605 ps: (cip_base_scoreboard.sv:249) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert fatal has unexpected timeout error
  UVM_INFO @ 310272605 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test otbn_controller_ispr_rdata_err has 2 failures.
  - 1.otbn_controller_ispr_rdata_err.31131107670441156424904471500832638376051632118561694170599869502880113729768
    Line 108, in log /nightly/runs/scratch/master/otbn-sim-xcelium/1.otbn_controller_ispr_rdata_err/latest/run.log
```
  UVM_ERROR @ 382038487 ps: (cip_base_scoreboard.sv:249) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert fatal has unexpected timeout error
  UVM_INFO @ 382038487 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
  - 4.otbn_controller_ispr_rdata_err.5580017653721694410935795854952642176395791364783890559466943644542096218781
    Line 113, in log /nightly/runs/scratch/master/otbn-sim-xcelium/4.otbn_controller_ispr_rdata_err/latest/run.log
```
  UVM_ERROR @  46205557 ps: (cip_base_scoreboard.sv:249) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert fatal has unexpected timeout error
  UVM_INFO @  46205557 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test otbn_alu_bignum_mod_err has 1 failures.
  - 3.otbn_alu_bignum_mod_err.27441242712254653491101541735057245907354624073745812325273905370445894513993
    Line 109, in log /nightly/runs/scratch/master/otbn-sim-xcelium/3.otbn_alu_bignum_mod_err/latest/run.log
```
  UVM_ERROR @  40619978 ps: (cip_base_scoreboard.sv:249) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert fatal has unexpected timeout error
  UVM_INFO @  40619978 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_*/rtl/otbn.sv,1383): Assertion ErrBitsKnown_A has failed has 3 failures:

Test otbn_sec_cm has 3 failures.

2.otbn_sec_cm.10489814442018566886298255205960061019860935588896663537749967813638000785015
Line 113, in log /nightly/runs/scratch/master/otbn-sim-xcelium/2.otbn_sec_cm/latest/run.log

  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1383): (time 65677121 PS) Assertion tb.dut.ErrBitsKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,948): (time 65677121 PS) Assertion tb.dut.u_otbn_core.DoneOKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,949): (time 65677121 PS) Assertion tb.dut.u_otbn_core.ImemReqOKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,950): (time 65677121 PS) Assertion tb.dut.u_otbn_core.ImemAddrOKnown_AKnownEnable has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,960): (time 65677121 PS) Assertion tb.dut.u_otbn_core.ErrBitsKnown_A has failed

3.otbn_sec_cm.110795739561122244226367112142961116933459672000291407838605950746541045137444
Line 124, in log /nightly/runs/scratch/master/otbn-sim-xcelium/3.otbn_sec_cm/latest/run.log

  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1383): (time 43051151 PS) Assertion tb.dut.ErrBitsKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,948): (time 43051151 PS) Assertion tb.dut.u_otbn_core.DoneOKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,949): (time 43051151 PS) Assertion tb.dut.u_otbn_core.ImemReqOKnown_A has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,950): (time 43051151 PS) Assertion tb.dut.u_otbn_core.ImemAddrOKnown_AKnownEnable has failed
  xmsim: *E,ASRTST (/nightly/runs/scratch/master/otbn-sim-xcelium/default/src/lowrisc_ip_otbn_0.1/rtl/otbn_core.sv,960): (time 43051151 PS) Assertion tb.dut.u_otbn_core.ErrBitsKnown_A has failed

... and 1 more failures.

UVM_ERROR (cip_base_vseq.sv:663) virtual_sequencer [otbn_imem_err_vseq] expect alert:fatal to fire has 1 failures:
- Test otbn_stress_all_with_rand_reset has 1 failures.
  - 2.otbn_stress_all_with_rand_reset.47644642451891537023325325660186321492485844064727894685756880395735138796213
    Line 230, in log /nightly/runs/scratch/master/otbn-sim-xcelium/2.otbn_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 237184847 ps: (cip_base_vseq.sv:663) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.otbn_imem_err_vseq] expect alert:fatal to fire
  UVM_INFO @ 237184847 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_base_vseq.sv:384) [otbn_reset_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 1 failures:
- Test otbn_reset has 1 failures.
  - 8.otbn_reset.9378685643287792166439604883807513624210318903469010521823595086817294292651
    Line 128, in log /nightly/runs/scratch/master/otbn-sim-xcelium/8.otbn_reset/latest/run.log
```
  UVM_FATAL @  83607558 ps: (otbn_base_vseq.sv:384) [uvm_test_top.env.virtual_sequencer.otbn_reset_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  83607558 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```