Blanket bomb a C source file with function entry/exit trace

'Blanket bomb' a file with trace for function entry / exit, from vim:

:%s/^{/{\r\ \ \ \ printf(">>>%s:%d\\n", __FUNCTION__,__LINE__);\r/
:%s/\(\ *\)\(return.*\n\)/\1printf("<<<%s:%d\\n", __FUNCTION__,__LINE__);\r\1\2/

or just using sed from the command line:

sed 's/^{/{\n\ \ \ \ printf(">>>%s:%d\\n", __FUNCTION__,__LINE__);\n/' Filename.c > Filename.c
sed 's/\(\ *\)\(return.*\n\)/\1printf("<<<%s:%d\\n", __FUNCTION__,__LINE__);\n\1\2/' Filename.c > Filename.c

How to use a 64bit machine to build & install a 32bit Linux kernel for another PC

I've often had the problem of having to build a Linux kernel for a slow machine, while having a much faster machine available. In my case, the slow machine is an Asus EEE-PC running 32bit Fedora, and my faster machine is an Intel i7 running 64bit Fedora.

In order for the faster 64bit machine to build a kernel and install it on the slower 32bit, you'll need to bear in mind three issues:

* How to build a 32bit kernel on a 64bit machine

* How to give access to the slower machine for installing the kernel

* How to actually install the kernel on the slower machine.

How to build a 32bit kernel on a 64bit machine

Assuming you already have kernel source and a suitable .config available on the faster machine, simply run your usual kernel make, with the addition of ARCH=x86 and with menuconfig as a target. In my case, this would be:

fast$: make ARCH=x86 -j16 O=/home/mark/Source/build-linux-2.6-32bit/ menuconfig

Then deselect the top option "[ ] 64-bit kernel" before exiting and saving. Now you can build the kernel with:

fast$: make ARCH=x86 -j16 O=/home/mark/Source/build-linux-2.6-32bit/

Note that the '-j' option gives the number of simultaneous jobs the make can handle, which is usually 2x the number of processors/cores in your system. I have 8 cores, so I use them all by specifying '-j16'.

The O= specifies the build directory where the build objects are placed. The .config you wish to use should go in here. There are many reasons why it is a good idea to have separate build and source dirs.

How to give access to the slower machine for installing the kernel

You could also use a Samba share for this, but I prefer using NFS. Note that this isn't the most secure method, using NFS in this way, but it works.

On the remote, slower, machine with NFS installed, export your root (/) and boot (/boot) directories by editing the /etc/exports file or using the system-config-nfs program. My /etc/exports file looks like this:

/boot *(rw,sync,no_root_squash)

/     *(rw,sync,no_root_squash)

Then restart the NFS services nfs. nfslock and rpcbind (portmap on older systems):

slow$: sudo service rpcbind restart

slow$: sudo service nfslock restart

slow$: sudo service nfs restart

Now it should be possible to remotely mount these exports on the faster machine, replace <remote-ip> by the name or IP address of the remote PC:

fast$: mkdir ~/remote-root

fast$: mkdir ~/remote-boot

fast$: sudo mount -t nfs <remote-ip>:/ ~/remote-root

fast$: sudo mount -t nfs <remote-ip>:/boot ~/remote-boot

Installing the kernel on the slower machine

Almost there, we just need to install the kernel image on the slower machine and reboot to run it!

For this to work seamlessly, a small patch is needed to the /sbin/installkernel script (on my Fedora machine, at least):

--- /sbin/installkernel

+++ /sbin/installkernel

@@ -32,7 +32,10 @@

cfgLoader=1

fi

-LINK_PATH=/boot

+if [ -z "$LINK_PATH" ]; then

+ LINK_PATH=/boot

+fi

+

RELATIVE_PATH=`echo "$INSTALL_PATH/" | sed "s|^$LINK_PATH/||"`

KERNEL_VERSION=$1

BOOTIMAGE=$2

Now we can export some environment variables to tell the kernel make install scripts where we would like our kernel and modules to be installed (Note the difference between remote-root and remote-boot!), while logged in as the superuser:

fast#: export INSTALL_PATH=/home/mark/remote-boot/

fast#: export LINK_PATH=/home/mark/remote-boot/

fast#: export INSTALL_MOD_PATH=/home/mark/remote-root/

For a Samba export, just replace the exported variables above with their equivalents (usually ~/.gvfs/root\ on\ remote/ or similar for Fedora).

Now we can simply run the install make targets on our fast machine, which installs to our slower, 32bit target PC:

fast#: make ARCH=x86 O=/home/mark/Source/build-linux-2.6-32bit/ modules_install install

Then edit your grub.conf (or equivalent, as I haven't yet found out why /sbin/new-kernel-pkg doesn't update this automagically on the remote machine).

Reboot, select the new kernel from the GRUB menu (or equivalent), and happy hacking!

Keeping it all in mind

Back in the 50's, the psychologist George A. Miller wrote a paper entitled 'The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information'. While quite a dry and involved read, it's findings (and other corroborating evidence) are of great significance to anyone creating a software design, architecture, process, checklist, API or code listing. In fact, anything where a large amount of items or concepts can be grouped or categorised may benefit from its findings, namely that:

There is a limit to the number of items we can keep in short-term memory - 7 plus or minus 2.

While not applicable to basic lists - where the reader doesn't simultaneously need to be aware of every item in the list - it is a powerful heuristic when applied to, for example, a software architecture layer. In this case a person trying to understand the architecture would benefit from awareness of all parts of the architecture at the level being studied, and in doing so finds the architecture more easily discoverable and transparent than if the layer were more finely partitioned.

Another great example comes in the form of a process description or checklist, especially where safety is critical. Here it is important not to miss any steps or checks, and the chance of doing so is reduced if the number of steps are kept to a manageable amount.

It's difficult to list everything to which this heuristic can be applied - there are always opportunities available if you can spot them. With practice, it can become second nature - like many other design heuristics (The appendix of 'The Art of Systems Architecting' by Maier and Rechtin is a great source for such things).

If you want some more examples, just take a look at some of the other posts in this blog... :o), but also next time you're wading through a mass of code, design information or an architecture description and finding it difficult then bear in mind how it is organised, is the author trying to make you 'bite off more than you can chew'?

This applies even more so if you are designing or developing. Your own software (especially open source) could benefit greatly if it's more easily discoverable, understandable and therefore maintainable by others who may not be so familiar with its workings.

Peer code reviews

Peer code reviews can have a devastating effect on software bugs, greatly reducing their number and scope. However, not all code reviews are the same and some are more effective than others. There is no one-shoe-fits-all guide to code reviewing, but there are ways of getting the right code review process for your situation.

At the bottom end of the effectiveness scale is a peer looking at a screen resplendent with the review code. This is better than no review, but still has drawbacks:

• The review procedure will vary according to the reviewer and their state of mind, some reviewers will be more picky about certain issues or even miss things others would find easily.
• Any problems found and commented on in the review may not be fixed - due to forgetfulness, other 'more important' pressing issues or thinking that they are trivial (to the author under review). Anyhow, the problems are probably not followed up afterwards to make sure they're resolved.
• A lack of reviewing guidelines can lead to some comments being taken a bit too personally.
• Most people when reviewing will only look through the code once from top to bottom, and pick out any issues of any type as they go along (syntax issues, loop counting and memory allocations for example). This is much less efficient than looking for one type of issue at a time, and scanning the code once for each type - if you don't believe me, try it!

In order to mitigate missing as many bugs as possible during a code review, the generally agreed best practice contains the following guidelines:

1. Use a checklist, some examples can be found in the links below.
2. Review for one type of issue at a time.
3. Improve the checklist to look for bugs which slip through the reviews as time goes on.
4. use code printouts on which you can scribble comments - I've not come across any software tool that is as effective as paper.
5. Log any issues not resolved during the review process, for example in the form of a bug database entry - even it does seem trivial now, experience has taught me that it might be the the most important thing in the world next month! It should also be possible to make the reviewer aware of any previous issues found with the code under review.
6. Have your process geared towards reviewing little and often.
7. Be balanced - make sure to note what the code does successfully, as well as it's issues and offer praise where it is due.

More thorough details and further reading can be found here:

• http://my.safaribooksonline.com/9780768685855/ch14lev1sec2?portal=informit
  
• http://en.wikipedia.org/wiki/Personal_Software_Process
• http://www.macadamian.com/index.php?option=com_content&task=view&id=27&Itemid=31
• http://smartbear.com/docs/BestPracticesForPeerCodeReview.pdf
• http://www.developer.com/tech/article.php/3579756

Easy SCM

Software Configuration Management (SCM) is an important part of software development - without robust SCM, no software development project can achieve a high quality result.
To this end, there are a few simple characteristics that indicate good SCM - Personally, I haven't encountered a successful project whose SCM plan does not have these characteristics:

1. Be a snail - leave a trail! Changes are never versioned without a documented reason.
2. Documented reasons are always either defect fixes, or software enhancements. Both defects and enhancements can be identified by an ID without the need to log detailed explanations for each change - all that is required is for the change to cross reference the defect or enhancement (from a bug database or project plan, for example).
3. Each change is made for a logical reason, and one reason only. Changes aren't combined or split over several different versions.
4. It is easy to 'do some archaeology' by quickly running older configurations. e.g. There exists a store of automated nightly builds with corresponding configuration IDs available.
5. The way of working is cultural, not 'enforced' - although it should be difficult to 'absent-mindedly' not follow the process. i.e. You can't make a change without a defect or enhancement ID to log against it.
6. The SCM system is transparent and visible, so others with an interest can view progress (although not necessarily take part in the activities - that can be damaging to progress!).
7. Each change represents no more than two man weeks work.

So, how does your project rate against these?

The IEEE software standards

The IEEE software standards is a very useful set of documents, even if you're not in an organisation that is into particularly formal software development methods. The standards do not necessarily have to be implemented to use the information they contain.
For example, they can be a good starting point if you're ever asked to plan or produce documents such as a software requirements specification (SRS), software configuration management plan (SCM plan), or a software architecture document (although for this I've found HP have available a much more usable document, "A Template for Documenting Software and Firmware Architectures").

The only downside is that they are not free, but many organizations have IEEE membership and have the standards available for employees.

Here's the list of IEEE standards and guides I've found to be of use, in no particular order, vaguely grouped for readability:


Project Management

• 1490 - Adoption of PMI Standard - A Guide to the Project Management Body of Knowledge (PMBOK) - A guide to project management knowledge and practices in general widespread use.
• 1045 - Standard for Software Productivity Metrics - A bit dated, but the practices and concepts it gives for measuring productivity are well described.
• 1058 - Standard for Software Project Management Plans - Worth a look if you're unsure as to what you'll need to put in a project management plan, but a little too specific to the IEEE way of doing things.
• ISO/IEC 12207.0 - Software Life Cycle Processes - Attempts to classify all the processes that contribute to software, and puts them into a framework.
• ISO/IEC 12207.1 - Software Life Cycle Processes - Life Cycle Data - Attempts to classify all the processes that contribute to software, and puts them into a framework.
• 982.1 - Standard Dictionary of Measures to Produce Reliable Software - A bit ancient (1988), but tries to give a description of measures that can be made on software and software projects that indicate the quality of the software, with all the mathematical rigour that involves.
• 1220 - Standard for Application and Management of the Systems Engineering Process - Similar to 1074, again use CMMI instead.

Software Engineering

• Guide to the Software Engineering body of Knowledge (SWEBOK) - Available here. A guide to software engineering knowledge and practices in general widespread use.
• 610 - Glossary of Software Engineering Terminology - A little dated, and leans towards IEEE understanding (as opposed to widespread understanding) of some terms, but can still be useful for reference, and is referenced by most other IEEE standards.
• 828 - Standard for Software Configuration Management Plans - If you need to produce an SCM plan and have nowhere to start, this will show you the way. Also discusses useful activities involved in managing and adhering to a SCM plan.
• 1002 - Taxonomy for Software Engineering Standards - Also quite ancient. A taxonomy is a method for classification, and this describes how a set of standards can be chosen to cover all necessary areas of software engineering.
• 1028 - Standard for Software Reviews - Gives criteria and practices for reviewing software - be it for development, acquisition or operation.
• 1061 - Standard for a Software Quality Metrics Methodology - Aimed at those measuring or assessing the quality of software, in a formal manner.
• 1074 - Standard for Developing Software Life Cycle Processes - Attempts to define a way of creating a good sw process. Not half as useful as CMMI.
• 1471 - Recommended Practice for Architectural Description of Software-intensive Systems - A version of Kruchen 4+1 for software architecture.
• 1042 - Guide to Software Configuration Management - Practices for performing SCM, and managing SC items within a project.

Quality

• 730.1 - Guide for Software Quality Assurance Planning - Great if you need to write and manage a Software Quality Assurance Plan, and have no idea of where to start - this lists and discusses the contents of such a document and good practices involved in managing it.
• 730 - Standard for Software Quality Assurance Plans - A lot more detailed than 730.1, and gives the format and content requirements a SQA plan should meet to conform to the IEEE standard.
• 830 - Recommended Practice for Software Requirements Specifications - Describes what should be contained in a good (albeit formal) SRS, and gives several example outlines of SRS documents.
• 1062 - Recommended Practice for Software Acquisition - Obtaining and using the right software, that's right for your needs is not an easy task. This gives some useful practices on performing this task.
• 1063 - Standard for Software User Documentation - Good practices for putting the relevant information into your user documentation.
• 1219 - Standard for Software Maintenance - This standard describes an iterative process for managing and executing software maintenance activities.
• 1228 - Standard for Software Safety Plans - Establishes criteria for the content of a software safety plan.
• 1233 - Guide for Developing System Requirements Specifications - A guide to obtaining and managing requirements in an SRS.

Testing

• 829 - Standard for Software Test Documentation - Gives a description of what should be in software test documentation (cases, logs, plans etc), and why. Gives the form and content of test documents, but does not say which documents are needed in particular situations.
• 1008 - Standard for Software Unit Testing - A standard for planning, building and executing unit tests.
• 1012 - Standard for Software Verification and Validation Plans - Gives a standard for V&V plans, describing what inputs, outputs and criteria are recommended for a project's V&V activities and should be recorded in a plan.
• 1044 - Guide to Classification of Software Anomalies - How to write and manage bug reports, very useful as in my experience, even some very experience software engineers have trouble in taking the time to write useful bug reports.
• 1059 - Guide for Software Verification and Validation Plans - Gives a process for using and managing V&V plans.

Design

• 1016.1 - Guide to Software Design Descriptions - Concentrates on documenting and using 'views' into a design, much like the Krutchen 4+1 paper.
• 1016 - Recommended Practice for Software Design Descriptions - How to go about writing a SDD, within the project life cycle.
• 1209 - Recommended Practice for the Evaluation and Selection of CASE Tools - CASE tools are notoriously difficult to choose and use successfully (see an earlier post), this tries to guide you around this particular minefield.
• 1320.1 - Standard for Functional Modelling Language - syntax and semantics for IDEF0 - A formal system/process modelling technique. Quite heavy, I've found ETVX to be much more usable.
• 1320.2 - Standard for Conceptual Modelling Language Syntax and Semantics for IDEF1X97 (IDEFobject) - Again, a formal system/process modelling technique. Quite heavy, I've found ETVX to be much more usable.
• 1348 - Recommended Practice for the Adoption of CASE Tools - Once you've got a CASE tool, the fun doesn't stop there. Carries on from 1209.

Can you sell a software process?

Most software groups (and I'm sure that this doesn't apply to just software groups) with any sort of history have a record of trying out prescribed process improvement after process improvement, many of them ending in some sort of failure - generally meaning that the process didn't meet the expectation or goals it was introduced with.

This can be especially true of process improvements that are built around an expensive software tool. I've had my fair share of colleagues lost to some sales type or evangelist touting the latest and greatest in software silver bullets. The use of the word 'evangelist' is telling, and conjures up images of an overzealous individual pushing unfounded ideas at you - it gives an indication of the mysic aura that a 'process guru' can give themselves by which one can be blinded.

CASE tools aren't that useful when they constrain you too much to one process, and some companies that push tools as well as processes rely on a self-created cult-like following whipped up from the guru of the day's scribblings. When these words of wisdom are unfounded, unproven and (even better) not understandable, they can be taken by savvy marketers to sell the tool.

A prescribed process (from a heavy one to a very light or 'agile' one) may fit perfectly into the company, work environment, or culture it grew up in but moving it into another environment, more often than not, can transform it into a useless or even dangerous beast.

In the translation, you can lose the meaning of important concepts, miss out on understanding vital assumptions and all the other pitfalls associated with one person attempting to describe a complex system to another. No matter how many reams of written documents or concise manifestos one writes, some things just get missed - even when the two communicator's work cultures are very similar.

What would be much more useful and less prone to failure is an expert in processes to mentor a group over a period of time (even indefinitely), who can prescribe and tailor a process to meet their current needs, to put in place mechanisms to measure the success and progress of the group and ultimately advance the group's software capability - a process specialist or change agent.

A software CASE tool is no substitute for an experienced mentor!