StockReader: Re-implement in Python

One of my first larger software projects was StockReader. That is a C++ web scraper for stocks.

The final result of StockReader

Now, I need to re-implement the program for four reasons: 

1. My firewall doesn't like blocks executables that sends several get requests
2. The source code is horrible and needs to be refactored completely in order to be more maintainable
3. I should be able to use regular expressions.
4. The program should be more automatic, selecting stocks automatically

I will not publish the program on Github, but I'll post some issues that I will discuss.

The first step will be to implement a script that gets a web page from an URL. From that html code, I'll extract the URLs for the key number sections for the stocks.

When creating code, it is crucial to understand what one is writing. It is important to resist the temptation to copy some code from stackoverflow and instead focus on really understand what one is writing.

In the Python documentation, there is a section about regular expressions.

TrafficControl: Locking Tracks for Trains - Tests and Visualization

After adding locks for tracks, I need to both visualize them and test them.

The tests are quite straight-forward, but they require some code.

• The first test case, trainListAddRmUpstream verifies that is it possible to add two trains in upstream on the same track, with a distance of more than 3000 m between the trains. It also verifies that the first train is removed first.
• The second tedst case, trainListAddDownstream verifies that a train that wants to enter a used track, where a train is moving in the opposite direction, will wait until the first train has left the track.

For the visualization, I'll change both model/view and the QML part.

Model/View:
A new column will be added for each track: Lock. The different states will be: Free, Start, End and Locked.

When something is changed for a track, the method Track::emitChangedSignal(trackID) is called. That method compiles a QStringList that contains the messages that shall be sent to the datamodel. The lock status is added as a string to the string list that is sent to the track data model. Also, a column is added for the data model.

Three tracks are locked. For example, dHja_Lun_E is locked at the end station (LundC). This means that trains can enter at Hjarup but not at LundC.

The next blog post will describe how I added the locks in the map.

TrafficControl: Locking Tracks for Trains - Code

Preventing trains to collide is an essential of any traffic controller. In this blog post, I'll implement some basic functionality for that purpose.

I'll use a simplistic model with two boolean variables: lockedUpStream (end station locked) and lockedDownStream (start station locked) with corresponding get and set classes.

When the train is in readyState (in a station), it will try to find a suitable track to the target station in Station::findLeavingTrackIndexToStation(int targetStationID).

If the train finds a suitable track and enters it, it will set a lock on the track:

• When a train enters a track in upstream direction, the lockedDownStream will be set to true.
• When a train enters a track in downstream  direction, the lockedUpStream will be set to true (see above). This happens in track::findLeavingTrackIndexToStation.
• When the last train leaves track in upstream direction, the lockedDownStream will be set to false.
• When the last train leaves track in downstream direction, the lockedUpStream will be set to false.

If all suitable tracks are locked, the train will wait one second and search for tracks again. 

As a safeguard, the track will also consider the locks when adding a train:

The locks are reset only if there is no train on the track.

The next blog post will cover tests and visualisation of track locks.

Future improvements:

For a certain time after a train enters the track, the track should be marked as red and no trains should be able to enter from any direction.

TrafficControl: Making User Interface Flexible with DockWidgets

Some readers may have seen that the user interface has a new look. Now, the user can select which widgets should be visible at run-time, how big they should be and where in the program they should be.

The user interface is designed in the file TrafficControl.ui, that is based on XML. That file contains the different widgets that makes up the user interface. 

When compiling, TrafficControl.ui generates a corresponding header file that  is called from the program (trafficControl.cpp/h):

The file ui_trafficControl.h is generated at runtime from the file trafficControl.ui

I've added qDockWidgets to the form that contains the elements. A qDockWidget is basically a container that can dock to different borders of the applications interface. The user interface assigns some space on the form, depending on the size of the contents, other widgets and the user's decisions.

A qDockWidget can be closed (set to invisible) and floating (presented in a separate window) if the user wants to. A list of the qDockWidgets is shown if the user right-clicks on the title for a qDockWidget.

This is needed as my program grows and there will be much more information to present to the user.

I've recently learnt how to use qDockWidgets and I'll explore more of its abilities in a future blog post. 

Future work: 

Add a panel of pushButtons, one for each qDockWidget that is available in the program. The buttons should have icons that describe the contents, for example Map, Train List, Basic Controls, Log window...

TrafficControl: Testing Asserts

Asserts are used to ensure that events that should never happen will never happen.

I've created a test case, where the program will try to add the same train to a station twice. If that happens, it would be a sign that the program has totally lost control of the network.

The program adds a train to a station. After that, it tries to add the same train to the station again. The expected result shall be an Assert.

When the code tries to add the same train again, the code asserts:

After this dialog, the test sequence stops and the remaining test cases are not run.

I found a workaround in a StackOverflow thread. A message handler will acknowledge the Assert message in the test case and the test suite can continue:

A message handler is specified before the expected Assert. It acknowledges the assert, without crashing the test.

TrafficControl: Opposite Direction TDD Case (2)

After fixing the first issue with trains that can't find adjacent stations, a new issue has been seen.

The train arrives to Gärsnäs and is supposed to enter the track Sim-GarW at end position (12000). It should move to position 0 and enter Simrishamn. However, it enters the track at start position (Simrishamn) and moves towards Gärsnäs. 

The train state model needs to handle the reversed scenario. A bool reversedTraffic with a get/set method is added to the track object. 

I made changes in the methods train::readyToRunning, train::runningState and train::runningToOpening. 

The train needs to consider its direction (normal or reversed) when entering a track.

train::runningState introduces a virtual position, that is

• virtualPosition = position, if train isn't travelling in reversed direction,
• virtualPosition = track length - position, if train is travelling in reversed direction

With these changes, the train is able to travel in reversed direction.

A crash was found for train::setTrackPosition(0). When it is updating the position, the method track::getCoordinatesFromPosition asserted if the position was 0, since it was trying to access coordinates at index -1.

TrafficControl: Opposite Direction TDD Case (1)

One big stub in trafficControl is that it currently only allows trains to travel on tracks in the intended direction (position 0 up to the end).

To enable trains to move in reversed directions, I will follow the Test-Driven-Development method.

The first iteration is to test a train on a station with one leaving reversed track. The track leaves Simrishamn (start) and arrives to Gärsnäs (end). A train in Gärsnäs that shall go to Simrishamn should find the track and travel in reverse on it to Simrishamn.

In this video, the train was supposed to travel back from Gärsnäs to Simrishamn, but that didn't happen.

The issue is that the method findLeavingTrackIndexToStation only searches for connected tracks where the end station is matching target station. For reversed tracks, the start station will match the target station. No suitable track is found and the function returns UNDEFINED.

To fix the issue, the program will search for a leaving track in the reversed direction in case it doesn't find a track in the ordinary direction.

A corresponding test case has been added:

With this change, the program still fails. After leaving Gärsnäs, the train enters the track near Simrishamn and travels to Gärsnäs. I'll  solve that in the next blog post.