SystemVerilog is a Big Mistake

I think we dropped the ball with SystemVerilog.
* It's based on old tech (but at least it has garbage collection). Why is it not more Python-like, y'know easier.
* It's a mishmash of languages
* It's getting 'unattainable'. For example, if you want to plug away at it on your own, there's no free simulator that you can practice with.

Toward a Fully Featured Programming Language

The Verilog standard should've only been updated to make it more useful from a HARDWARE DESCRIPTION point of view. SystemVerilog is an effort to grow Verilog towards a more traditional OOP programming language - and that's what's back to front. We should've taken Python (yield) (or even Go - after all it's built around concurrency and it compiles PDQ (not TCL, please)) and grown it to include a Verilog DUT.
SV adds useful stuff like hashes and foreach loops that make it a lot more expressive - stuff that's empiricaly proven to increase productivity by 100.09%. But why not just start from a real programming language in that case? It's not like OOP testbenches do connectivity and timing like traditional RTL - SV testbenches expect you to call .run() on all your class instantiations and pass around handles to interfaces for connectivity. And since we're back to forking a load of .run() methods, why not start from a 'real' programming language, and allow it to twiddle the inputs of RTL descriptions of hardware?

Adding Broken Things

Since SV is a huge amalgamation of things by an amalgamation of vested interests, things were added to the SV standard that should not have been.

program Block Fail

Also, what's with the program blocks? That's a fail right there. And we still have problems with time -0 initialisation, still have possible race conditions at the start of a sim if you want a monitor module to have reasonable defaults, and then change them at the start of an initial block.

final Blocks

I don't get these. They're supposed to be able to let you do things at the end of the simulation. But like most Verilog procedural blocks, you've no visibility on the order that they'll execute. So say you want to open a file at the end of a simulation and have all your testbench monitors write their status to it. Yay, so put a final block in each of your monitor blocks to write to the file... uh, hold on, how do you know that file has been opened? How do you keep the order consistent? Ah, I know, call a .summary() function/method for each of your monitors. But now to call these functions you need to know what monitors you have, so monitors have to register themselves somewhere because SV has no introspection. So now you've a single final block calling a bunch of .summary() functions and if you've only one final block, what's the point? You may as well just have a function that you call at the end of your 'main()' initial procedure.

Open Verification? Hmmm...

SV testbench-building methodologies seem to be settling around the UVM - a nice 'open' standard that's being put together by the Accellera consortium. Yeah, you can download the code for free and have a peek at it, and maybe send some patches back to fix things that trouble you, but it ain't open, baby. If you have to pay loads of cash for a simulator to run this, I'm not sure that you can claim that it's open.
This is another good reason for going the {Real_Programming_Language, Verilog} route. With just a Verilog-2001 open source simulator, open source programming language and some tasty interfacing, you'd be able to run fancy testbenches on pre-existing RTL from the comfort of your own home. No expensive licenses needed. And more than that, you wouldn't have to limit the maximum concurrent jobs on the compute farm to 10 when doing regressions because co-workers write pleading e-mails to you not to hog the licenses...

Assertions, Coverage & Constrained Randomisation

I admit that I haven't used assertions, coverpoints or constrained randomisation in anger. And I suspect that this weakens my argument somewhat. But this could be done in a Python module instead of, y'know, bolting together several existing languages? I've a feeling I underestimate the amount of work needed to get all this stuff working. Yip, I admit it - this portion of my argument is weak.

Companies

Companies. Why would they do {Real_Programming_Language, Verilog} when they could build SystemVerilog to steer us away from the opensource verilog simulators that were somewhat catching up, and make us all move to something that we need to look on feature vs price matrices to see which portions of the bright new thing we can afford to run? Companies, I suppose I can have nothing against them, after all I do work for one! They have to make a buck, I suppose.

So...

It's interesting to think about what a "Real Programming Language + Verilog 2001" SystemVerilog would look like. What Real Programming Language would we use? Would it actually improve productivity?

That Wiki Thing...

It's been roughly a year since my pet wiki has been active on the company's intranet. It's definitely been useful, but I think it hasn't completely lived up to the hopes and dreams I had for it.

Usefulness to My Good Self

As I'd planned, I've been using it as a kinda design notebook, although I still scribble on real paper as it's the quickest way to record thoughts. When I write a wiki page, I find I write for an audience other than myself. And that's no bad thing as I have to state assumptions and 'formally' defend any assertions. I'm convinced this is ok; my paper notebook is for exploration and the wiki is the crystalisation of the thought process that lead to the final design. The wiki is the definitive source of information about a topic, not a discussion. The wiki has added a sense of rigor to thinking behind the stuff I produce.

Y'know, maybe I shouldn't be setting up wiki pages willy-nilly. I shouldn't actually be doing my design in wiki pages. Wiki pages are supposed to be solid information, not cloudy half-thought-out explorations. It should not really be an extension of my paper notebook, should it?

Usefulness to My Teammates

This is harder to judge. I think it's somewhat useful to my teammates in a read-only sense, but that it's still considered as "Marty's wiki" and not "the wiki" as I'd hoped.
I have made an effort to let people know of its existence. After I complete a body of work, I check that the page in the wiki is reasonably accurate and then the link is sent around in the 'announcement' email. For example:

Hi All, I'm finished setting up the co-sim environment for our latest chip (which is the bee's knees, BTW, and going to make our company millions). See here (http://ourgroupswiki.some.address.com/) for info on the environment and instructions for launching a sim

That sort of thing. And there is evidence that people read it, but they don't edit it if something's amiss. I do get the odd query on the accuracy of instructions, but my teammates never change the information themselves. Maybe they've better things to be doing - maybe they don't feel that they're an expert in that field so need consensus. Who knows?

The Elephant in the Room - Sharepoint

The wiki's relationship with Sharepoint is still mostly undefined.

Sharepoint is our company's blessed online collaboration thingy. But it's become a dumping ground for powerpoints and word documents. And mostly Office 2007 versions of stuff I've no hope of opening on my linux workstation (vendor lock-in, much?). Rant aside, this is where the latest datasheets, latest marketing info, latest formal design documents go. And to be honest, it's probably the correct place for that info.

So...

I need to properly define the wiki's place in the grand scheme of things. I know it has one, but I haven't yet been able to articulate it. I also need to ask my teammates why it's not "the wiki" yet.

I dunno why I'm invested in this so much.

Canonical Signed Digit Representation

I've recently had the opportunity to play around with multiplierless filter designs. Here's some Python code to convert numbers to and from Canonical Signed Digit (CSD) representation. It does fractional too, as I like to keep track of my binary points with negative net indices in Verilog-land.

It's based on a short paper I can't remember the name of. More specifically, it's based on the pictures from a short paper I can't remember the name of as I couldn't really follow all the set theory in the text.

To use it, put it on your path somewhere and:

canavan% python 
Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15) 
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import csd
>>> csd.to_csd(34)
'+000+0'
>>> csd.to_csd(34.75)
'+00+0-'
>>> csd.to_csd(34.75,4)
'+00+0-.0-00'
>>> csd.to_csd(34.75,6)
'+00+0-.0-0000'
>>> csd.to_decimal('+0000')
16.0
>>> csd.to_decimal('+0000.0-000+0-000+0000-')
15.761955261230469
>>> 

Beware, I don't do any input validation yet...

Oh yeah, linkage: http://sourceforge.net/projects/pycsd/

I Program Computers

A new housemate moved in recently. We were getting to know each other - talking about our backgrounds, our favourite football teams and all the usual getting-to-know-you good stuff. He'd half remembered from our initial meeting that I did something vaguely technical for a living, and asked did I "program computers or what?".
"I'm an electronics engineer. I help to design the digital parts of chips", I NACKed.
"Ah", says he, "so how do you do that then?"
"Emm", I was caught out. "By ah, programming computers...", I sheepishly admitted.
It brings up a topic close to my heart - are Electronic Engineers (EEs) learning as much as they should from Computer Science and Software Engineering?

Digital Design is Programming

Software Engineering is important to EEs because digital designers, and especially functional verification engineers, are in essence specialised software engineers. For digital designers, our thoughts are necessarily grounded in hardware but those thoughts are expressed in software. The special requirements of concurrency and timing for describing hardware requires dedicated Hardware Description Languages (HDLs), but these are programming languages none the less - computers can be made to execute them.

If computers can run our HDLs as programs, then its natural as engineers to want to check the arse off our designs before they make it to manufacturing. We want to make sure that we've expressed our ideas correctly. We're obsessive about checking so we put our functional verification engineer hats on and we run simulations, and now we're suddenly programming for real. Our testbenches and testcases are now software proper. It no longer matters if the code we write is translatable into flip-flops and NOR gates, so long as the input signals are wiggled in the correct way and that the outputs wiggle as we'd expect. And even better (maybe?), we're allowed to abstract now.

I'm of the opinion that a lot of Electronic Engineers don't read enough about software development as they should. Software seems to be, or at least seemed to be, a minor detail that we could get the co-op to sort out. And as far as my own university course was concerned - why did I have to independently discover the joys of source control? I've read a few books like "Code Complete", "Emergent Design" and "Pragmatic Programmer" and wished with every line I read that an equivalent existed for us digital designers. Maybe there is, it's just that programming related resources are easier to find on the web.

Since we're all programmers now, we should learn how to program. From what I read, real software programmers seem to have a small niggling worry that they're somehow inferior to 'real' engineers. That's backwards though - us 'real' engineers need to start befriending real programmers and learn from them. We're so dependent on computers that we need to learn how to program for real. We need source control, we need unit tests, we need to learn to refactor and we need to learn to spot code smells. We need to write scripts to generate RTL, scripts to launch batches of sims over the network and create Makefiles to automate synthesis. We're software engineers and we haven't the slightest clue we are - at least, we've no ideal we will be when we leave college.

Drawing Circuit Diagrams - Update

Well, after a bit of wrangling with the EGB layout algorithm - things are working out!



There are still a few crossovers on the outputs of U8 & U9 which I haven't got to the bottom of yet...

Animation

To help get to the bottom of such things, I've implemented a bit of animation to show me how the layout is progressing at each step. Using python's generators to unroll the main layout loop was the key here. First, the circuit data structure is drawn, then after a small delay the .next() is called on the generator, and the circuit redrawn. This continues until the generator is spent. Pretty nifty if I do say so myself...

Improvements

At the minute, the layout algorithm is sweeping from the inputs of the circuit to the outputs. I'm worried that this won't be optimum for untangling all types of circuits. So, once I debug my EGB algorithm implementation, I'll experiment with the following to see what gives the best results:

• inputs to outputs


• outputs to inputs


• inputs to outputs to inputs

I'm also concerned about the initial state of the circuit data structures. Maybe I've giving the algorithm too easy of a time. The instantiations in the circuit data structure are more or less in the order that they are in the verilog file. Maybe I should mix-up the instantiation order in the verilog files. Or maybe have a switch to randomize the instantiation orders in the data structure...

I've also to trawl/profile the code and look for optimizations...

Next Steps

After playing around with the layout algorithm, I think I'll add a final stage to tidy up the drawing of the nets. Once I get something half-pretty going, I'll concentrate on parsing a bigger subset of the verilog language.

Wikis

Keeping Engineering Info in Wikis

I'm starting an experiment at work. I want to liberate the design notebook. And this revolution will be wikified.

If my group 'published' sections of their notebooks on an intranet wiki, I'm convinced we'd see lots of benefits. I realise that having a paper trail is very important for patents and such, but I don't envisage the wiki replacing the notebooks - rather that the entries in the wiki would be a somewhat more polished version of the more interesting and useful scribbles.

What goes in the wiki?

Lots of things. Solutions to weird bugs. Testbench documentation. I'd even suggest that an entry for every major new block in each project goes into the wiki, with the important legacy stuff added as we go along. The block info would detail how the block works and more interestingly, why the block is. More info than in an email introducing the block, but maybe less info than for a design review. Even technical questions in emails to you could serve as topic pointers for a wiki page.

I think it's the wrong place for sim results. It's wrong for block pin lists or schematics. It's probably the wrong place for anything which has to be copied from other sources to keep it up to date.

Benefits of Wikization

The obvious benefit is that all this stuff which normally lives in only one or two people's heads or inboxes is available to and searchable by the entire group. Another benefit is that writing an entry in the wiki should focus the designer, making them think more about what they're doing which should help increase the quality of our designs. It would also be a ready-made source for info and text for design review documents, datasheets, customer presentations and the like.

Resistance

There are a few drawbacks, though. The main one is getting buy-in form the rest of the team. I'm not naive - I know that if I tried to get it decreed that everyone has to use the wiki in the way I outline, it would raise eyebrows, roll eyes and be dismissed as another layer of red tape and beaureaucy.

I have a plan* though - I'm going to lead by example and people will see the revolution as righteous. I've started to put interesting stuff in the wiki and I'm starting to point team members to it when they come looking for info. They're eventually going to start thinking, "Hmm, Marty would know that, I check that wiki thing of his before I ask him". This is going to be cool for a while until they spot an error, at which time I'm going to lightly suggest that they get themselves an account and fix it up. They're going to see the benefits of the wiki and start adding information themselves and things will get cooler. OK, there was a leap of faith there, but there's no harm in trying it out.

Aside: We've also a Sharepoint site too, but this seems to be a place for dumping documents and todo lists. I'm going to have to think a bit more about how the wiki fits with it.

Initial Wiki Usage Observations

So. The wiki I'm using at the moment is Wikimedia, because that's what sysadmin kindly set up for me. I like the way it stores edit history. I am finding it useful.

The main problem I see is with engineering diagrams. There's no stable drawing plugin for that species of wiki. State diagrams and example timing diagrams have to be created elsewhere and uploaded as .pngs or whatever to the wiki. I don't like the fact that the master document for the diagrams is elsewhere, making it difficult for others to correct or append them. And even if there were a stable drawing plugin, would the drawings be of high enough quality to use in more formal documents?

I think diagrams are important in engineering documentation, and would love if the barrier for entering diagrams into the wiki was lowered. I'd love if we all had graphics tablets (or tablet PCs) and could just scribble a quick diagram only for it to appear in the wiki. I'm contradicting myself here a little, but if it's a tossup between no diagrams because its a pain in the arse to get them in the wiki and sketchy diagrams that need to be redrawn with more care for more formal documents, then sketchy wins for me all the time. I like diagrams...

Future

I think the future of our group has a wiki in it. Lets see how the experiment goes...

Resources

• twiki


• wikimedia

* OK, it's not really my plan, I robbed it from http://www.randsinrepose.com/ , or more specifically, his book "Managing Humans"

Verilog RTL Decommenter

We're transferring a bit of soft IP to a customer, and decided to remove all the comments from the RTL files. Our IP is protected by an NDA, so we decided against obfuscation as we felt this may cause unnecessary hassle if we're asked to help debug the IP integration. We did decide to remove comments so that any stray profanity, "FIXME"s and "This is an ugly, ugly hack but..."s are not presented to the customer. It was also an opportunity to include a copyright header to the RTL file, too.

It fell to me to script the removal of the comments. Being a bit of a python fan, I went searching for some pythonic regexp-based comment remover. I found a C decommenter here, but it needed a few modifications to work with verilog comments which I present below.

#! /usr/bin/env python

# remove_comments.py
import re

def remove_comments(text):
  """ remove c-style comments.
      text: blob of text with comments (can include newlines)
      returns: text with comments removed
  """

  pattern = r"""
                          ##  --------- COMMENT ---------
         /\*              ##  Start of /* ... */ comment
         [^*]*\*+         ##  Non-* followed by 1-or-more *'s
         (                ##  group 1
           [^/*][^*]*\*+  ##
         )*               ##  0-or-more things which don't start with /
                          ##    but do end with '*'
         /                ##  End of /* ... */ comment
         |                ##  -OR-
         //[^\n]*         ##  // comment to end of line
       |                  ##  -OR-  various things which aren't comments:
         (                ##  group 2
                          ##  ------ " ... " STRING ------
           "              ##  Start of " ... " string
           (              ##
             \\.          ##  Escaped char
           |              ##  -OR-
             [^"\\]       ##  Non "\ characters
           )*             ##
           "              ##  End of " ... " string
         |                ##  -OR-
                          ##
                          ##  ------ ANYTHING ELSE -------
           .              ##  Anything other char
           [^/"'\\]*      ##  Chars which doesn't start a comment, string
         )                ##    or escape
"""

  regex = re.compile(pattern, re.VERBOSE|re.MULTILINE|re.DOTALL)
  noncomments = [m.group(2) for m in regex.finditer(text) if m.group(2)]

  return "".join(noncomments)


copyright = """//  --------------------------------------------------------------
//
//  My Company Inc.  - Confidential Information
//  Copyright 2005-2008
//
//  --------------------------------------------------------------"""

if __name__ == '__main__':
  import sys
  filename = sys.argv[1]
  code_w_comments = open(filename).read()
  code_wo_comments = remove_comments(code_w_comments)

  #fh = open(filename+".nocomments", "w")
  #fh.write(code_wo_comments)
  #fh.close()

  print copyright
  print code_wo_comments

First of all, I added a bit to the regexp to spot one-line comments that start with // - as mentioned in the perl FAQ - see the emphasised section in the above code.

I also got rid of the single quote string matching section of the regexp because verilog doesn't have such strings. It was also accidentally matching the code between two number specifiers which prevented the removal of the comments in what it thought was a string. For example, the comment below would not be removed:

assign a = 1'b0;
// Some comment
assign b = 1'b1;

The regexp itself saves two groups; group 1 is comment group and group 2 is a non-comment group. Printing group 2 is the thing to do if you want the comments removed. If the regexp matches a comment, then group 1 is text and group 2 is empty - printing group 2 effectively "removes" the comment. If the regexp matches a non-comment, then group 2 is text we want to keep, so we print it.

This decommenter script is used as part of an overall script which prepares our code for handover. The RTL is exported from our CVS directory, decommented and tar.gz'd - ready for secure FTPing to our customer...