Tuesday, May 27, 2008

Constrained Random Verification

After stewing for a bit on constrained random verification, it's beginning to loose a bit of its sheen. Let me explain...

The first question is: What gets randomised? Well, there are two types of inputs to our chips: control and data, ignoring supplies. So let's think about what randomizing control and data inputs might entail.

Randomizing Control Inputs

For control input we can randomise timing, order or address-data pairs. Randomising the timing between control writes caught bugs for us in the past, so we find this useful. Randomising the order of control writes doesn't make sense for us as we give customers specific powerup sequences to avoid various unwanted transients.

Throwing constrained random address-data pairs at the chip seems like A Good Thing, but there's a lot of infrastructure needed to get at the full benefits. At the minimum you'll need a high-level model of your chip against which you can check the behaviour of your chip. But the very point of high-level models is that they are not as complicated as the chip itself. I worry in this case that we'll end up designing each chip twice - once in RTL and once as a model. I may be getting confused here, so I should try to gather my thoughts on high-level modelling at a later time.

Randomizing Data Inputs

I'm failing to see the benefits of randomizing the input to datapaths. I've issues with the high-level modelling again, and anyway truely random data is nonsense when piped through filters! (GIGO). So how would constrained random data look like? Usual signals with noise on top? 'Usual signals' is what we're trying to do away with though... (Could I use that trick where you can set a maximum dx/dt?)

I'm not sure what constrained random input signals would look like in our case. And I'm not sure what type of errors they could catch in the datapath (assuming we already stress them with types of signals that we know can over-range our sums).

Random Chip Configurations

Maybe I'm thinking about this at too low a level. Maybe we should be randomizing the configurations of our chips. For example, our serial data ports can work in a variety of modes: I2S, LJ, RJ etc. We've sims to check the correct functionality of each of these serial formats. But when it comes to other sims, for example, checking out the DAC signal chain, we usually feed it with data in the default serial format (I2S). Maybe it's things like serial formats and number of DACs powered up that should be randomised? Maybe that's a bad example as the interfaces between our serial ports and the rest of the chip are well defined?

Conclusion

I haven't come to one, really - the jury's looking to get put up in a plush hotel. I might explore the randomisation of our chips' configurations and maybe make sure we're stressing our datapaths. And I haven't even touched upon functional coverage, which if I'm not careful, could fall prey to the same traps as code coverage.

Friday, May 16, 2008

SystemVerilog

I've just come back from a week-long SystemVerilog course, presented by one of the folks at Doulos. The course was, I'd have to admit, very interesting and extreemly well delivered - J_ certainly knew his stuff. There seems to be a lot of cool features in SystemVerilog, and other slightly underwhelming stuff, that I want to rant about.

A Fistfull of Features...

SystemVerilog is basically Verilog 2001 with a shedload of new ideas, features and keywords, system tasks, mini-languages, etc, etc. Although there are one or two new language features to make your RTL look prettier, to my mind the majority of the shiny new things are for verification engineers.

The Good

I'm mostly a verification engineer, and SystemVerilog offers me 3 huge and genuinely exciting powers that I want to try out right away; these being assertions, constrained random testing and functional coverage.

Assertions

Assertions are great for making sure your design does what you wanted it to do. They can check the value of a signal or two at a point in time. But more interestingly, by using a regular-expression type mini-language, they can also check signal behaviour during a sequence of clock cycles.

The idea is that you sprinkle assertions all around your RTL in interesting places (synthesis tools will ignore them), and they'll let you know if whatever they're monitoring steps out of line. They'll also help you get to the source of a bug far quicker than a traditional chip-as-a-black-box testbench setup - in which case you have to wait for bugs to propagate to the outputs, then follow the chain of events back to the bug.

Another win for assertions is when you code up a module, and a colleague ends up using it. If the module has assertions on its inputs, it can complain if it is not being fed with the correct signals. Now any bugs that are reported to you are real bugs, and your time is not wasted with bugs due to a misunderstanding of the module's input specs.

Functional Coverage

Functional is a new angle on design verification. Currently our verification plans consists of a big list of sims that must pass before we can tape out. This list is mostly derived from the specs - we go through the specs and try to write a sim testcase that will cover each bit of functionality.

Functional coverage is different because first up, you describe to the simulator every bit of functionality that you want to see. Then it tells you what behaviours in your list it has encountered during the course of a sim (coverage results are usually aggregated over a bunch of sims). If you're careful when writing the functionality descriptions, you can say that functional verification is finished when 100% of targets are hit!

The big payoff for functional coverage is when it's used with constrained random testing.

Constrained Random Testing

Constrained random testing makes it possible to trade verification engineer brain cycles for CPU cycles. It involves throwing random-yet-tuned stimulus at your design, shaking the innards of the chip in more ways than any verification engineer could engineer given a reasonable amount of time.

The fun starts when assertions are added to our design and our list of functional coverage points has been defined. Instead of tuning bunches of testcases to exercise each behaviour, we can just run a few randomised testbenches for longer and let luck stumble across all our behaviours. (Could we breed testcases?) Of course, purely random stimulus is not going to be helpful here due to the GIGO principle, hence we guide or constrain the randomness. And we're probably going to need a bus functional model to check the outputs of our design too.

The Bad and The Ugly

This is where I descend into rant mode, so be warned...

The Tower of Babel

Nothing in SystemVerilog is new under the sun, everything has been magpied from elsewhere: assertions are based on Sugar; OOP sort of follows C++; and other bits and pieces from OpenVera, Superlog and other things I can't quite remember. This leaves the whole SystemVerilog thing looking a bit un-integrated (or uneven, or inconsistent) to me. In most places you use begin-end, other places you use curly brackets. In most places you finish lines off with a semi-colon, in constraint blocks you don't. While in some cases this is not too bad and is perfectly understandable (for example, the PSL), for the most part it just feels a bit of a hodge-podge mish-mash in places.

Classes

OK, I can see how this might get slightly controversial, and maybe this is more related to the inconsistency I've already noted, but I think that object orientated programming has been kludged into SystemVerilog. And it's an ugly kludge.

The amount of hoops that need to be jumped through just to use a class in SystemVerilog that wiggles a few pins, and keeping it reusable is crazy! First, define an interface, give it a clocking block and a modport, throw the handle to the interface all around the place, instanciate your classes, interfaces, and DUT, and probably a few other things that I've forgotten too.

OK, so you only have to do all this once, but it seems ugly and unintuitive. The concurrency that's an essential feature of an HDL is lost for classes and has to be regained again by forking a .run() method on all your classes. The connectivity that's an essential feature of an HDL is lost and has to be faked by using references to interfaces though which pins are accessed. Crazy.

Could us poor hardware engineers not be introduced to the benefits of object orientated programming in a gentler way? Why can't modules not be our 'classes', and be inhierited as well as being instantiated? I've a nagging feeling that I'm missing something huge about the way OOP needs to be implemented, and that maybe it had to be done that way - I'd love to know why.

A Few Features More


Overall, I'm genuinely excited by some of the possiblities that SystemVerilog has opened up to our verification setups. I plan to try out some of these things in our current testbench and report on the progress - I won't be changing it to a class-based architecture any time soon though!