Programming - Part 2

This is by far the most important part of any project, whether you're designing an air-traffic control system, a global ordering system, or a personal website... You can't solve the problem unless you understand it (and if you try - you'll probably make it worse!).

The analysis is ironically the one bit that often gets overlooked for two reasons:

1. People often think they understand a problem (or requirement) - even though they don't.
   
   

   (I've been to many system design meetings where people couldn't resolve an issue - because they were trying to solve the symptoms of a problem, without understanding the problem itself. It often takes a fresh pair of eyes to solve an old problem!)
   
   

2. The next phases - design and coding - are more tangible.
   
   

   It's easy to get straight into writing bits of code and trying them out, or specifying features. Some people feel they aren't being productive unless they're issuing orders or writing code - it always takes time to understand things, but it saves time in the long run.
   
   

All systems at their most basic have three components: inputs, processing, and outputs.

Systems exist in order to solve a problem. Most computer systems are designed to solve problems for people, e.g. 'how to keep track of all my customers', or, 'how to respond to emails automatically'.

When analysing any system, especially complex business systems, you should look at each problem and ask yourself: "Is this a genuine problem, or is it a symptom of a more fundamental flaw?" Make sure you're solving the right problem before you start.

So, the analysis phase is to understand the problem. From this you'll write a list of requirements. These are the tasks that need to be done to solve the problem.

Example: The problem is security for a program you've written. You only want registered users logging in, so you have a 'login' dialog box. As you think about it, the following list of requirements emerges: Inputs: username and password, a 'submit' button. Processing: When 'submit' is pressed, check the details against the database... ...If the details are OK - (Output:) the user is taken to the next screen ...If the username doesn't exist, or the password is wrong, allow 3 re-tries. ...If all re-tries have been used - disallow the user from logging in for 30 minutes. Outputs: It will need some method of telling the user if the username / password was wrong... A status line perhaps? It should look nice. You may well come up with some questions: Should it record all login attempts? Are usernames case-sensitive (is 'BOB' the same as 'bob')? As you answer these questions, you refine your analysis, until all the major problems have been addressed. Then you can move on to the design phase.

You should be able to determine all the requirements, and how you'll meet them, before writing a single line of code!

All the information you have gathered about the system you're writing should be documented. Everywhere's different, but a 'Functional Design Specification' document is usually written which describes the analysis and design stage of the project.

Once you have your list of requirements, you can think of ways to provide those things in your program.

This is the stage where you do your brainstorming - to come up with the best solutions and algorithms that will meet your requirements.

A large part of the design used to be the platform and programming tools. With the web, there's still a choice, but the choice of Unix Webhosting, Perl, PHP and SQL is the simplest to learn, and it's the cheapest too. You can do anything you could ever want with these tools from the smallest to the largest project, no problem.

The web is a very nice straightforward programming environment! It also largely dictates which bits are written in which languages:

• Visible web pages are written in HTML
• Programs are written in Perl or PHP (these are both general-purpose languages)
• Databases are created and accessed with SQL

In general terms, the rest of the design process, however big the project, goes like this:

• You define the inputs:
  ...all the types of data you need to store, and where they come from.
  
  
• You define the processing:
  ...What do you want to do with that data. Reformatting, storage, combining...
  
  
• You define the outputs:
  ...How do you want the results to be displayed. On the screen, on a printer, as sounds...

For example, the automated order processing we'll be using has:

Inputs - data types:

• orders - from the payment processing company.
• products - database table maintained by you.

Processing:

• Check and Store orders in the orders table.
• Get the password for the product from the database and send it to the customer.
• Log all activities - especially errors.

Outputs:

• A new order in the database.
• Send password (to access product) in email to customer.
• Inform webmaster of any errors
• Write a log file.

In a commercial project, the final part of the design would often be to define the names of everything (the variables, functions etc) and exactly how it works. Sometimes you'd use a flow diagram, sometime pseudocode to illustrate a particular algorithm you're using. (Algorithm: a step-by-step method for solving a problem).

If your project is at all complicated I'd recommend using flow diagrams - they are a wonderful tool for visualising process flow. Refer back to the Tools module for Flow Diagramming software.

At this stage, most of the design is complete. We know the platform and languages, we know our requirements, and what were going to do with the data, we just need to know how. This is where knowledge of your chosen language comes in!

The best way to learn a system?... Read the manual! Until you know what's available in a language, you should spend some time reading through the lists of functions that are available to you. All programming languages come with a function reference. Don't waste your time spending hours writing a function, to later find one similar (better) already exists.

This is where you actually write your source code. You try bits out as you go - so you can find and remove any obvious errors.

As you write your code, you should always add lots of relevant comments - specially demarkated text which is ignored by the interpreter/compiler etc. Comments remind you what the code's doing, how and why. Comments are essential - especially if the code's complicated. They are there to help you read your own code. Use comments to break your code up into manageable, understandable chunks. Later, if you get bugs, you can find any part of the program quickly, without having to read any code.

Comments in HTML look like this:
 <!-- comment text between the tags! -->

Comments in Perl:
 # the whole line is a comment...

As you code your program, it's common practice to start simple and test it as you go. You start with the simple tasks that can be done on their own, and get them working before progressing. As you become more experienced, you can write larger blocks of code before needing to test them, but you will experience bugs in your code. It's inevitable, don't get upset, don't take it personally...

Once you think your current bit of code is ready, you run it and see what happens... If it does what you expect, then you'd carry on and write the next bit. If it doesn't, you try to find out why & fix it. This is 'debugging'.

There are two types of errors that will occur:

• Syntax errors are errors of grammar, i.e. you missed a semi-colon, or a bracket. These errors will prevent the program running at all as they are caught by the language pre-processor.
  
  
• Semantic errors are errors of meaning. i.e. where the program is doing something you didn't intend it to.

Syntax errors are usually quite easy to fix because the language pre-processor tells you what it thinks is wrong. i.e. 'Error(22): semi-colon missing line 29'. Sometimes you can spend a while looking for a lost curly bracket '{' for example, but it's never that hard.

Semantic errors are the real bugs, but there are plenty of tools available to find them, and even the simplest method is usually adequate...

The first thing you should do if you have a bug is to re-read your code. It's quite likely you'll see straight away why the computer has done what it has. You may have the wrong variable in a certain place, the order of things may be wrong, perhaps you used the wrong function or operator somewhere...

For example: in many languages, the assignment operator is '=', but the equals operator is '=='... The result of this is that the following statement always returns TRUE whatever the value of $a because you're actually assigning '1' to $a, not comparing them:
 if ($a=1) return TRUE; else return FALSE;

If after you've checked your code, you still can't see what's wrong, it's time to find out what's going on inside the running program...

The simplest way to debug things is using the language's built in print or echo function to write text to the screen. Sometimes a simple statement to show where you got to in the program before it crashed is all you need, i.e.:
 print 'Got Here';

or,
 ... initialisation code ...
 print 'Initialisation done.';
 ... code to get data from database ...
 print 'Got data from DB';
 ...etc...

It's fairly common to have functions in a program that are used exclusively for debugging. For example, if you are writing a program which displays graphical charts of data, you'd often write a function which simply prints the data out to the screen as text, so you can a) check the data values are processed correctly, then b) compare the output of the graph drawing function against the data.

Common debugging measures include:

• At the beginning of every function, print the values that were passed to the function. i.e.
   sub check_password (username, password) {
  print "User:", username, "Password:", password;
  ... rest of function code ...
  
  
• Print out the values of variables at strategic points in the program. i.e.
   $b = myfunction($a);
 if (debug) print $a, $b;
  
  
• Write functions to create test data. You can use random number functions to create huge databases of test data that you can then set your program to work on.
  
  

Obviously, after every run, you spend time reading and understanding the output. It rarely takes long to find & fix bugs once you have the knowledge, and tools.

There are special programs for debugging called debuggers they can be very useful, allowing you to:

• Stop and restart the program at any point.
  
  
• Stop the program if something happens (like a value goes over a limit).
  
  
• Examine or change any variable.
  
  
• See exactly where a program crashed.
  
  
• Step through the program a statement at a time.
  
  
• Some even allow you to run your program backwards!

However, you never really need to use a separate debugger. They do speed up debugging for professional programers, but there's a bit of a learning curve. I don't think there's a PHP debugger available.

Build your programs up in small functional steps and test them as you go. Use plenty of comments. If you have a serious problem, use plenty of 'print' statements to find it. Don't be frightened of writing lots of debugging code, you can always keep it out of the final 'production' version of your program.

The only way to learn to program is to do it - same as everything. Once you've completed the course you'll have done enough to be understand all the concepts involved and with all the references, be completely self-sufficient!

Different from the debugging you do while programming; the formal testing stage is started only once you're quite sure that the program is perfect... It invariably isn't. There's no denying it, this is the dullest part of programming. Like analysis, it's often overlooked, but it's still very necessary if you want to sleep at night.

As computers are so stupid, and take everything so literally, they frequently don't work as expected. Flaws in a program are 'bugs' and like programming itself, can be infinitely varied. Bugs can range from the program not starting, stopping or crashing, to just dropping a single digit from an obscure data item.

Don't worry about causing your webhosting computer to crash, unlike some operating systems, it's incredibly difficult to make Unix crash by writing bad code. Unix will usually just give you an error message when a program fails.

Bugs can emerge from software being used in an unforeseen way. Some people seem to have a knack of finding bugs. You can spend hours testing a program when someone who has never seen it before will crash it in 5 seconds!

In business, you always have to test your own code. The newer you are to programming - the more bugs there will be in your code. There will always be bugs in newly written code!

You will first test to see all the requirements are met. You then test for all the things that you can possibly think of that could go wrong. i.e. inappropriate usage, missing or corrupt data, system or other failures and so on. (There are limits though; you don't have to test the language itself!)

To use the password entry screen we discussed above as an example, then in addition to ensuring it looks OK, you'd test the following cases (conditions):

Test Cases:

Correct username & password entered. Correct username, incorrect password. Incorrect username. Blank username / password. 3 failed tries Capital letters - does it handle them correctly. Does it handle spaces before the username / password.

You should write all your test cases in a test document. Then you write a test that confirms that the program works as expected for each case. Then you run the tests and record the result: e.g.

Test Results:

Case 1: correct username and password entered. Expected Result: access to main program granted. Test Result: OK Case 2: correct username, incorrect password. Expected Result: login screen reappears, with 'password invalid, attempt 1.' in status line. Test Result: OK Case 3: incorrect username. Expected Result: login screen reappears, with 'username invalid, attempt 2.' in status line. Test Result: OK Case 4: blank username / password. Expected Result: login screen reappears, with 'username invalid, attempt 3. Login disabled for 30 minutes!' in status line. Test Result: OK Case 5: 3 Failed tries Expected Result: cannot login for 30 minutes Test Result: OK Result of test: OK

If you do bother with a formal test, you may be surprised at how often you'll find things wrong with your programs! It's tempting to think that the debugging done during the coding process is enough, but it rarely is.

A good test should test every path through the code. I.e. it should test every single line of code.

Once your testing is complete - the program is officially finished, ready, done! Give it a version number and back it up somewhere!

Computers and Programming General
Encyclopedia.com	have a large section on computers.
You Should Learn To Program.	An online book on programming with humour.

Computer Languages
Computer Programming Languages.	A (huge) list of all the programming languages ever!
A Dictionary of Programming Languages	An, er, Dictionary of Programming Languages!
Paradigms -- Architecture of Programming Languages.	This is a technical and in-depth look at language architecture. Loads of info, but not for the faint of heart.

Programmers Resources
FreeCode.com	A large repository of free source code.
Developer Shed	Open Source and Web Development in general
WeberDev	Web Development Resources
WebMonkey	Web Development Resources, Tutorials.
CProgramming.com	A collection of C and C++ related resources.
Programmingtutorials.com	online tutorials.
ProgrammingTips.com	Computer Programming Tips, Tricks, Hints and Secrets.
Sticky Sauce	The webmasters development resource - everything you need to design, build, develop and promote your web site.

Miscellaneous Resources
Writing Good Pseudocode.	An article on pseudocode.
The Object-Oriented Page, by Ricardo Devis	An article on object orientation.
Computer Chess Programming.	All about programming chess.
Programming Artificial Intelligence.	An online magazine with free and non-free sections.
PC Mechanic.	Lots of info and history about PC's and hardware.