Thursday, October 16, 2008

Verilog RTL Decommenter

We're transferring a bit of soft IP to a customer, and decided to remove all the comments from the RTL files. Our IP is protected by an NDA, so we decided against obfuscation as we felt this may cause unnecessary hassle if we're asked to help debug the IP integration. We did decide to remove comments so that any stray profanity, "FIXME"s and "This is an ugly, ugly hack but..."s are not presented to the customer. It was also an opportunity to include a copyright header to the RTL file, too.

It fell to me to script the removal of the comments. Being a bit of a python fan, I went searching for some pythonic regexp-based comment remover. I found a C decommenter here, but it needed a few modifications to work with verilog comments which I present below.

#! /usr/bin/env python

# remove_comments.py
import re

def remove_comments(text):
""" remove c-style comments.
text: blob of text with comments (can include newlines)
returns: text with comments removed
"""

pattern = r"""
## --------- COMMENT ---------
/\* ## Start of /* ... */ comment
[^*]*\*+ ## Non-* followed by 1-or-more *'s
( ## group 1
[^/*][^*]*\*+ ##
)* ## 0-or-more things which don't start with /
## but do end with '*'
/ ## End of /* ... */ comment
| ## -OR-
//[^\n]* ## // comment to end of line
| ## -OR- various things which aren't comments:
( ## group 2
## ------ " ... " STRING ------
" ## Start of " ... " string
( ##
\\. ## Escaped char
| ## -OR-
[^"\\] ## Non "\ characters
)* ##
" ## End of " ... " string
| ## -OR-
##
## ------ ANYTHING ELSE -------
. ## Anything other char
[^/"'\\]* ## Chars which doesn't start a comment, string
) ## or escape
"""

regex = re.compile(pattern, re.VERBOSE|re.MULTILINE|re.DOTALL)
noncomments = [m.group(2) for m in regex.finditer(text) if m.group(2)]

return "".join(noncomments)


copyright = """// --------------------------------------------------------------
//
// My Company Inc. - Confidential Information
// Copyright 2005-2008
//
// --------------------------------------------------------------"""

if __name__ == '__main__':
import sys
filename = sys.argv[1]
code_w_comments = open(filename).read()
code_wo_comments = remove_comments(code_w_comments)

#fh = open(filename+".nocomments", "w")
#fh.write(code_wo_comments)
#fh.close()

print copyright
print code_wo_comments


First of all, I added a bit to the regexp to spot one-line comments that start with // - as mentioned in the perl FAQ - see the emphasised section in the above code.

I also got rid of the single quote string matching section of the regexp because verilog doesn't have such strings. It was also accidentally matching the code between two number specifiers which prevented the removal of the comments in what it thought was a string. For example, the comment below would not be removed:
assign a = 1'b0;
// Some comment
assign b = 1'b1;

The regexp itself saves two groups; group 1 is comment group and group 2 is a non-comment group. Printing group 2 is the thing to do if you want the comments removed. If the regexp matches a comment, then group 1 is text and group 2 is empty - printing group 2 effectively "removes" the comment. If the regexp matches a non-comment, then group 2 is text we want to keep, so we print it.

This decommenter script is used as part of an overall script which prepares our code for handover. The RTL is exported from our CVS directory, decommented and tar.gz'd - ready for secure FTPing to our customer...

Friday, July 11, 2008

Fixed-Point Arithmetic with Verilog

I'm doing a bit of hardware RTL at the minute, which is a change from my usual testbench code. I'm trying to implement a datapath using fixed-point arithmetic and I'm finding that verilog is not helping me as much as I thought it would. And I've come to realise something...

Fixed-point arithmetic in verilog is broken. And that makes me sad.

Representation

First things first. Let's try to represent a fixed-point number in verilog. What about a vector of bits? Cool, let's say we'll represent our fixed point numbers in N bits:

We'll allow M bits for the integer part and F for the fractional (and, of course, M+F = N).
reg [M+F-1:0] my_number;
OK, so far so good. This is not too self-documenting though - how can you tell that this is a fixed-point number? And where is its binary point? Even worse, you can't tell verilog that the number is supposed to be fixed-point - it doesn't even have a fixed-point 'type'.

Another way is to signal a fixed-point number by having the fractional bits have negative indexes:
reg [M-1:-F] my_number;
For example, an M=1 and F=4 number's vector would be indexed [0:-4]. In this case, we can decree that the binary point is always between indexes 0 and -1, and that any vectors declared with negative indices is a floating point number. This also has the nice property that the value of each bit in the vector is 2index, just like the integer representation.

It's just bookkeeping, though. Nomatter how we spin it, we can't really get verilog to help us out with our fixed-point numbers. For example, say we wanted to add a 1.4 number to a 2.5 - (we'd expect a 3.5 result...):
reg [0:-4] a;
reg [1:-5] b;
reg [3:-5] c;

always @(*) begin
// c = a + b; // this won't line up the binary points for us
c = {a, 1'b0} + b; // we have to make sure that the binary points line up ourselves
end

In this case, Verilog won't line up the binary points for us, it'll line the vectors up LSB to LSB. We're left to make sure that we pad whichever vector to line up the binary points.

Display

Since we can't tell verilog we're working with fixed-point numbers, they're not going to be displayed correctly. Any $displays in the testbench are going to display integers. "But wait!", I hear you say - sure couldn't you just write a function to properly display your fixed-point numbers? Not easily. Functions can't be parameterised (as fair as I'm aware), so you'll have to write conversion functions for each different size of fixed-point number to be displayed. The reason is because slices of vectors must have constant expressions: you can writemy_number[-1:F] if F is a parameter, but not if F is a variable.

In a waveform viewer, our fixed-point numbers are going to be displayed as integers as well. Unless we write expressions (in SimVision, anyway) to convert them.

So, What Now?

To recap, you can't easily work with fixed-point numbers in verilog. Verilog can't help with lining up the binary point for arithmetic, and fixed-point numbers are display incorrectly both in $displays and in waveform viewers.

Should verilog support fixed-point arithmetic? Could you do something with structs and operator overloading in SystemVerilog? (Maybe not for synthesis). It turns out that I don't have solutions or recommendations for any of this, so this was just a rant. Sorry about that...

All this does mean that a lot of the high-level datapath design must be done in Matlab or whatever. I hope Matlab has fixed-point libraries...



An Aside: Text Macros

Why do none of my simulators happily accept the following?

`define two_lsbs(a) a[1:0]
module mess();
reg [3:0] some_vector;
initial begin
$display( "%b", `two_lsbs(some_vector) ); // OK
$display( "%b", `two_lsbs(5) ); // broken
end
endmodule

When you go to use this, simulators complain about unmatched parenthesis when a numeric literal is supplied. Why?


.

Tuesday, June 3, 2008

Drawing Circuit Diagrams

If you've been browsing some of my previous posts, you'll know that I'm interested in writing an open source tool to generate schematics from some Verilog RTL. And you'll also probably remember that I was trying to come up with the layout & routing algorithms for the schematics myself.
I'm also failing miserably, you may remember. This is as far as I got with the genetic algorithm layout before I decided to abandon it on speed and reproducability grounds:


So, I've honoured the pragmatic promise I made to myself, and I've turned to the interwebs for help.


Vocabulary

Drawing automated pictures of relationships in Computer Science goes by the name of Graph Drawing, a branch of Graph Theory. According to this stuff, I'm looking to draw Layered Orthogonal Directed Graphs:
  • 'Layered' from the fact that I can arrange the instances into columns. Sugiyama seems to be the main man when it comes to algorithms for this sort of graph.
  • 'Orthogonal' because I want the nets to go in right-angles.
  • 'Directed' because there's a flow in the drawing. For us EEs, this flow is left to right, but in graph theory it's usually top to bottom. So my problem would've been with the x-placement.
In graph theory, my RTL module instantiations are nodes and nets are edges.

Existing Code

The first thing I did with my new-found pragmatism was to look for open-sourced code I could rob use. Preferably this code would be a C/C++ library (for speed) with Python bindings (for handiness), but I'd settle for pure Python. I didn't find exactly what I was looking for; either there was a lack of examples and screenshots, no python bindings, or the library was closed source. That said, if I'm willing to learn SWIG to create python bindings, or I'm willing to create my own examples, there are a few libraries to investigate further:Some of the proprietary stuff could've been exactly what I need: tomsayer.com (sorry, this tries to resize your browser window) had a teaser of a circuit diagram, and yFiles had intrigingly-named ChannelEdgeRouter class.

Even if none of the above open source libraries end up suiting my project, at least I have the freedom to look at the code and study the algorithms they use when cooking my own.


Literature Search

Then I stuck a whole pile of terms into the search engine to see what turned up. I tried various combinations of terms including 'graph', 'drawing', 'routing', 'layout', 'channel', 'layered', '2d' etc. and added more as they turned up. Although I got some useful introductory slide decks from university courses, I did bang my head up against sites such as ieeexplore and springerlinks which expected me to pay for stuff.

The searching did throw up a pair of papers by Eschbach, Günther & Becker which seem promising. One of which, Orthogonal Circuit Visualization Improved by Merging the Placement and Routing Phases, especially so.


Homework

I think the next stage of my endeavour is to read the papers by EGB (hehe, Eternal Golden Braid) I mentioned above, and have a look into those graph drawing libraries, maybe igraph seems the most appealing at a first cut.

Tuesday, May 27, 2008

Constrained Random Verification

After stewing for a bit on constrained random verification, it's beginning to loose a bit of its sheen. Let me explain...

The first question is: What gets randomised? Well, there are two types of inputs to our chips: control and data, ignoring supplies. So let's think about what randomizing control and data inputs might entail.

Randomizing Control Inputs

For control input we can randomise timing, order or address-data pairs. Randomising the timing between control writes caught bugs for us in the past, so we find this useful. Randomising the order of control writes doesn't make sense for us as we give customers specific powerup sequences to avoid various unwanted transients.

Throwing constrained random address-data pairs at the chip seems like A Good Thing, but there's a lot of infrastructure needed to get at the full benefits. At the minimum you'll need a high-level model of your chip against which you can check the behaviour of your chip. But the very point of high-level models is that they are not as complicated as the chip itself. I worry in this case that we'll end up designing each chip twice - once in RTL and once as a model. I may be getting confused here, so I should try to gather my thoughts on high-level modelling at a later time.

Randomizing Data Inputs

I'm failing to see the benefits of randomizing the input to datapaths. I've issues with the high-level modelling again, and anyway truely random data is nonsense when piped through filters! (GIGO). So how would constrained random data look like? Usual signals with noise on top? 'Usual signals' is what we're trying to do away with though... (Could I use that trick where you can set a maximum dx/dt?)

I'm not sure what constrained random input signals would look like in our case. And I'm not sure what type of errors they could catch in the datapath (assuming we already stress them with types of signals that we know can over-range our sums).

Random Chip Configurations

Maybe I'm thinking about this at too low a level. Maybe we should be randomizing the configurations of our chips. For example, our serial data ports can work in a variety of modes: I2S, LJ, RJ etc. We've sims to check the correct functionality of each of these serial formats. But when it comes to other sims, for example, checking out the DAC signal chain, we usually feed it with data in the default serial format (I2S). Maybe it's things like serial formats and number of DACs powered up that should be randomised? Maybe that's a bad example as the interfaces between our serial ports and the rest of the chip are well defined?

Conclusion

I haven't come to one, really - the jury's looking to get put up in a plush hotel. I might explore the randomisation of our chips' configurations and maybe make sure we're stressing our datapaths. And I haven't even touched upon functional coverage, which if I'm not careful, could fall prey to the same traps as code coverage.

Friday, May 16, 2008

SystemVerilog

I've just come back from a week-long SystemVerilog course, presented by one of the folks at Doulos. The course was, I'd have to admit, very interesting and extreemly well delivered - J_ certainly knew his stuff. There seems to be a lot of cool features in SystemVerilog, and other slightly underwhelming stuff, that I want to rant about.

A Fistfull of Features...

SystemVerilog is basically Verilog 2001 with a shedload of new ideas, features and keywords, system tasks, mini-languages, etc, etc. Although there are one or two new language features to make your RTL look prettier, to my mind the majority of the shiny new things are for verification engineers.

The Good

I'm mostly a verification engineer, and SystemVerilog offers me 3 huge and genuinely exciting powers that I want to try out right away; these being assertions, constrained random testing and functional coverage.

Assertions

Assertions are great for making sure your design does what you wanted it to do. They can check the value of a signal or two at a point in time. But more interestingly, by using a regular-expression type mini-language, they can also check signal behaviour during a sequence of clock cycles.

The idea is that you sprinkle assertions all around your RTL in interesting places (synthesis tools will ignore them), and they'll let you know if whatever they're monitoring steps out of line. They'll also help you get to the source of a bug far quicker than a traditional chip-as-a-black-box testbench setup - in which case you have to wait for bugs to propagate to the outputs, then follow the chain of events back to the bug.

Another win for assertions is when you code up a module, and a colleague ends up using it. If the module has assertions on its inputs, it can complain if it is not being fed with the correct signals. Now any bugs that are reported to you are real bugs, and your time is not wasted with bugs due to a misunderstanding of the module's input specs.

Functional Coverage

Functional is a new angle on design verification. Currently our verification plans consists of a big list of sims that must pass before we can tape out. This list is mostly derived from the specs - we go through the specs and try to write a sim testcase that will cover each bit of functionality.

Functional coverage is different because first up, you describe to the simulator every bit of functionality that you want to see. Then it tells you what behaviours in your list it has encountered during the course of a sim (coverage results are usually aggregated over a bunch of sims). If you're careful when writing the functionality descriptions, you can say that functional verification is finished when 100% of targets are hit!

The big payoff for functional coverage is when it's used with constrained random testing.

Constrained Random Testing

Constrained random testing makes it possible to trade verification engineer brain cycles for CPU cycles. It involves throwing random-yet-tuned stimulus at your design, shaking the innards of the chip in more ways than any verification engineer could engineer given a reasonable amount of time.

The fun starts when assertions are added to our design and our list of functional coverage points has been defined. Instead of tuning bunches of testcases to exercise each behaviour, we can just run a few randomised testbenches for longer and let luck stumble across all our behaviours. (Could we breed testcases?) Of course, purely random stimulus is not going to be helpful here due to the GIGO principle, hence we guide or constrain the randomness. And we're probably going to need a bus functional model to check the outputs of our design too.

The Bad and The Ugly

This is where I descend into rant mode, so be warned...

The Tower of Babel

Nothing in SystemVerilog is new under the sun, everything has been magpied from elsewhere: assertions are based on Sugar; OOP sort of follows C++; and other bits and pieces from OpenVera, Superlog and other things I can't quite remember. This leaves the whole SystemVerilog thing looking a bit un-integrated (or uneven, or inconsistent) to me. In most places you use begin-end, other places you use curly brackets. In most places you finish lines off with a semi-colon, in constraint blocks you don't. While in some cases this is not too bad and is perfectly understandable (for example, the PSL), for the most part it just feels a bit of a hodge-podge mish-mash in places.

Classes

OK, I can see how this might get slightly controversial, and maybe this is more related to the inconsistency I've already noted, but I think that object orientated programming has been kludged into SystemVerilog. And it's an ugly kludge.

The amount of hoops that need to be jumped through just to use a class in SystemVerilog that wiggles a few pins, and keeping it reusable is crazy! First, define an interface, give it a clocking block and a modport, throw the handle to the interface all around the place, instanciate your classes, interfaces, and DUT, and probably a few other things that I've forgotten too.

OK, so you only have to do all this once, but it seems ugly and unintuitive. The concurrency that's an essential feature of an HDL is lost for classes and has to be regained again by forking a .run() method on all your classes. The connectivity that's an essential feature of an HDL is lost and has to be faked by using references to interfaces though which pins are accessed. Crazy.

Could us poor hardware engineers not be introduced to the benefits of object orientated programming in a gentler way? Why can't modules not be our 'classes', and be inhierited as well as being instantiated? I've a nagging feeling that I'm missing something huge about the way OOP needs to be implemented, and that maybe it had to be done that way - I'd love to know why.

A Few Features More


Overall, I'm genuinely excited by some of the possiblities that SystemVerilog has opened up to our verification setups. I plan to try out some of these things in our current testbench and report on the progress - I won't be changing it to a class-based architecture any time soon though!

Wednesday, March 5, 2008

Some Verilog Tips & Tricks

I thought I'd share a few Verilog tips & tricks I discovered recently that help when you're trying to build a simulation that doesn't care where it's run from in your directory structure.

Gather Files & Environmental Variables

Gather files are list of simulator commands that are included using the -f flag. You know this, but you may have a different name for them. In these gather files I list all the Verilog module files that I'm using in the simulation, as well as the include directories needed. I specify each file relatively (eg ../../../) from a simulation base directory ($BASEDIR). I actively avoid absolute paths: this gives designers the freedom to set up their simulations anywhere; and the simulations can be run from any of our company's sites. So long, of course, if the designer checks the code out from our versioning system...

run_sim.sh:
#! /usr/bin/sh
export BASEDIR="../../../" # setenv in csh

simulator -f "${BASEDIR}/config/sim.gather"


${BASEDIR}/config/sim.gather:
//
// TITLE: A Simulation Gather File
//
+incdir+${BASEDIR}/block1
${BASEDIR}/block1/rtl/module1.v
${BASEDIR}/block1/rtl/module2.v
${BASEDIR}/block2/rtl/module1.v
${BASEDIR}/block2/rtl/module2.v

The magic here is the 'export' command in the shell script. For best results, the script calling the simulator can calculate what this should be based on the current working directory ($PWD). Verilog & Debussy will correctly substitute any environmental variables it sees in gather files. Our designs are fairly complicated, and I've used nested gather files sucessfully to mimic Verilog-2001's 'Configurations'. But don't get me started on using Verilog configurations with NC-Verilog and Debussy...

Specifying bitmap files for $readmemb and $readmemh

The next problem is making any bitmap files for $readmem tasks portable. The main problem here is that macros are not expanded within strings, nor are you allowed to split strings with them. For example, the code below won't work:

`define BASEDIR ../../../
`define BASEDIR_STR "../../../ // may not even get this far...

module test();
reg [3:0] mem [3:0];

initial begin
$readmemh("`BASEDIR/rom.dat"); // macro won't be expanded
$readmemb(`BASEDIR_STR.rom.dat"); // won't be accepted by simulator - split over string
end

endmodule

I came up with a solution using a reg vector to hold a strings and then using $sformat, but it seemed ugly and I thought there must be a better way. And I found it using concatenations.
module test();
reg [3:0] mem [3:0];

initial begin
$readmemb({`BASEDIR_STR,"/rom.dat"}); // string literal concatenation!
end

endmodule


This works like a charm so long as `BASEDIR_STR is a string literal - "../" is ok but just ../ is not. To keep things directory-agnostic, I pass the base directory string as an argument to the simulator, eg (escaping the quotes is important here to get the path passed to the simulator as a string literal):

run_sim.sh:
#! /usr/bin/sh
export BASEDIR="../../../" # setenv in csh

simulator -f \
"${BASEDIR}/config/sim.gather" \
+define+BASEDIR_STR=\"${BASEDIR}\"