Skip to content

Issues and Testing

Slides

Software development in long-running projects is often organized around issues. After reading this you will…

  • understand how issues, version control and testing interact
  • understand two levels of test driven development
  • understand browser driven development

What is an issue? It’s a unit of work. You might know it as a “user story”, a “work item”, or by other names.

At the top level, we only look at the issue itself:

  1. Define the Issue: The problem (bug) or new requirement (feature/story) is given a clear description
  2. A developer picks up the issue
  3. It is implemented (we will zoom in on this step later)
  4. It is prepared for manual testing (we will zoom in on this step later)
  5. Now the implementation can be tested in the browser, and compared to the issue
  6. If the implementation is accepted, the issue is closed, otherwise return to step 2

Let’s look at a simple example issue from a Laravel project:

Issue #3: For admin users, show a Dashboard at '/admin' and display the number of Stamps

It is very short and simple. Maybe it is the first issue, and other issues build on it? Maybe there are issues #4, #5 and #6 that add more data to the dashboard?

Maybe the developers created this issue when they broke up a longer user story into smaller pieces.

The next level is the git workflow:

  1. after picking the issue, the developer creates a feature branch and a merge request. The merge request is set to “draft” status.
  2. then the issue is implemented (we will zoom in on this step later)
  3. when the developer thinks the implementation is done, they remove the “draft” status
  4. now another developer does a code review
  5. if changes are needed, we go back to step 2.
  6. now the feature branch can be deployed to the staging server, where it is available for manual testing.
  7. if problems are found in manual testing, we go back to step 2.
  8. if the implementation is accepted, the feature branch can be merged and deployed to production

To create the feature branch, the developer picks a name and creates the branch with git on the command line:

git checkout -b feature_dashboard_issue_3

Your team might have a convention for naming branches.

The branch can be pushed to the git server immediately:

git push origin feature_dashboard_issue_3

You create the merge request in the web interface of the git server. When you push a new branch to GitLab, the reply even contains the URL for this:

remote: To create a merge request for feature_dashboard_issue_3, visit:
remote: https://gitlab.example.com/groupname/projectname/-/merge_requests/new?merge_request%5Bsource_branch%5D=feature_dashboard_issue_3

This is what creating a merge request looks like. The screenshot shows two important details:

  • be sure to activate the checkbox “Mark as draft”
  • in the description, you can refer to issues by their number

The merge request compares your feature branch to the main branch. You can add commits now or later on - the merge request will always show the latest version.

When we zoom in on step 2, “the issue is implemented”, we find the cycle of test driven development (TDD): we write the test first, and then the code. For an issue that delivers value to the user, we start with feature tests:

  1. break the issue up into a list of possible feature tests - but don’t implement anything yet
  2. write the simplest feature test from your list. If you run your test suite now, the new test fails: it is “red”
  3. implement just enough to make the new test (and all the existing tests) pass: they are “green”
  4. commit and push
  5. refactor if you think it’s necessary. all the tests still pass.
  6. commit and push
  7. are you done with the whole issue? if not go back to step 1

Let’s look at the example issue:

Issue #3: For admin users, show a Dashboard at '/admin' and display the number of Stamps

There are several aspects that we could turn into feature tests:

  1. Only admin users should have access to the URL ‘/admin’
  2. I expect the heading “Dashboard” to be on the page
  3. I expect the number of stamps to be on the page

Which one is the simplest to implement? Probably number 2.

Let’s look for a place to put the feature test. Take a look at the files tests/Feature/*Test.php.

Maybe we should create a new test file for the Dashboard. There is a generator that could do that:

php artisan make:test AdminDashboardTest

This creates the file tests/Feature/AdminDashboardTest.php with an example test in it.

Or you could copy an existing file with feature tests, and use the existing code as inspiration.

The dashboard is only meant for admin users. In this project, these are the users with the status STAFF. So our test logs in as a staff member with actingAs() before it requests the page:

// file tests/Feature/AdminDashboardTest.php
use App\Models\User;
it('shows the heading Dashboard', function () {
$staff = User::factory()->create(['status' => 'STAFF']);
$this->actingAs($staff)
->get('/admin')
->assertOk()
->assertSee('Dashboard');
});

When running the feature tests in the terminal:

php artisan test

the new test will fail, because there is no page at ‘/admin’ yet:

FAILED Tests\Feature\AdminDashboardTest > it shows the heading Dashboard
Expected response status code [200] but received 404.

There are different ways to implement this feature. In this simple case, “browser driven development” works well.

To get a feature test to pass, we sometimes need to build a new web page, or a new view in an MVC system. Then it makes sense to switch to “browser driven development”: you reload the page in the browser after every small step, and the next error message tells you what to do next.

By using the browser in every step, we make sure that all the parts of the system fit together. Finally we reach a stage where the feature test passes.

This also works for new REST API endpoints. There, the goal is to see the right JSON output in the browser instead of a web page.

This is how it looks in Laravel.

Open the (new) URL ‘/admin’ in the browser. You see the first error:

404 Not Found

Laravel does not tell us more here: there is simply no route for this URL.

To define the missing route, we edit the file routes/web.php. Read the existing routes and find a good place for the new route.

The admin page is not connected to one specific model, so it does not belong in a resource controller like the EventController. We will use a new PageController for pages like this:

// file routes/web.php
use App\Http\Controllers\PageController;
// ...
Route::resource('events', EventController::class);
Route::get('/admin', [PageController::class, 'admin'])->name('admin'); // new route

Now reload the page ‘/admin’ to see the new error:

Illuminate\Contracts\Container\BindingResolutionException
Target class [App\Http\Controllers\PageController] does not exist.

So the route exists, but the controller it points to does not. Let’s create it with artisan:

php artisan make:controller PageController

Reload the page ‘/admin’ again to see the next error:

Error
Call to undefined method App\Http\Controllers\PageController::admin()

So the route and the PageController exist, but there is no action admin there.

Let’s go to app/Http/Controllers/PageController.php and define a new method admin:

use Illuminate\View\View;
class PageController extends Controller
{
public function admin(): View
{
return view('admin');
}
}

Now reload the page ‘/admin’ again to see the new error:

InvalidArgumentException
View [admin] not found.

The error message tells us which view is missing: view('admin') looks for the file resources/views/admin.blade.php.

Let’s create this file:

{{-- file resources/views/admin.blade.php --}}
<x-layout title="Admin Dashboard">
<h1>Admin Dashboard</h1>
<p>...nothing here yet...</p>
</x-layout>

This version of the code already fulfills the feature test we wrote. So we are done with “Browser Driven Development”.

Now is a good time to commit and push!

To get a feature test to pass, we sometimes need to add code to several classes. In this case it makes sense to do an “inner cycle” of TDD. It works like the TDD cycle with feature tests, but this time we write unit tests:

  1. Think of all the parts you might need to fulfill the feature test, and how you would test each of them - but don’t implement anything yet
  2. pick the simplest part, and the simplest unit test for it, and write that unit test. (red)
  3. implement it (green)
  4. commit and push
  5. refactor if you think it’s necessary. all the tests still pass.
  6. do you have all the pieces needed to get the feature test to pass? if not, go back to 1

In this “inner cycle” you build one piece after the other, until you have everything the feature test needs.

Let’s go back to the “list of possible feature tests” in our example:

  1. Only admin users should have access to the URL ‘/admin’
  2. I expect the heading “Dashboard” to be on the page
  3. I expect the number of stamps to be on the page

Number 2 is already implemented.

Let’s pick Number 3, write a feature test for it, and then think about what we need to implement it.

// file tests/Feature/AdminDashboardTest.php
use App\Models\Stamp;
use App\Models\User;
// ...
it('displays the number of stamps', function () {
$staff = User::factory()->create(['status' => 'STAFF']);
Stamp::factory()->count(2)->create();
$this->actingAs($staff)
->get('/admin')
->assertSee('2 stamps in total');
});

The test database is empty at the start of every test, so we know exactly how many stamps there are: the two we just created with the factory.

The project contains a model Stamp. So we could use Eloquent to count the number of stamps: Stamp::count().

Be careful not to write Stamp::all()->count(): that gives the same number, but it loads every single stamp from the database into PHP, and then counts them in PHP. Stamp::count() lets the database do the counting with SELECT COUNT(*).

We could put this code into the view:

<h1>Admin Dashboard</h1>
<p>{{ \App\Models\Stamp::count() }} stamps in total.</p>

Or we could put the code in the controller, and then display it in the view:

class PageController extends Controller
{
public function admin(): View
{
$numberOfStamps = Stamp::count();
return view('admin', compact('numberOfStamps'));
}
}

or we could put the code in the Stamp model, and just call a method in the controller:

class PageController extends Controller
{
public function admin(): View
{
$numberOfStamps = Stamp::totalNumber();
return view('admin', compact('numberOfStamps'));
}
}

The last version is best: the Stamp model should contain the business logic of stamps. Right now the logic is simple. But later there could be stamps that are not approved yet, and should not be counted. Or there could be stamps that are invalid or outdated. Deciding which stamps count is business logic, so it belongs in the model.

Let’s write a unit test for the method of the Stamp model. If the project already has a file tests/Unit/Models/StampTest.php, we can just add a new test case there. Otherwise we create it with:

php artisan make:test Models/StampTest --unit

This unit test needs the database. In a new Laravel app the tests in tests/Unit do not start Laravel, so they cannot use the database or factories. Check tests/Pest.php: 'Unit' has to be in the list of folders that use the TestCase (see The Test Environment).

// file tests/Unit/Models/StampTest.php
use App\Models\Stamp;
it('knows how many stamps there are in total', function () {
expect(Stamp::totalNumber())->toBe(0);
});

This test expects that there are no stamps at all.

That is correct: the test database is empty at the start of every single test. RefreshDatabase runs the migrations once, and then wraps every test in a database transaction that is rolled back at the end of the test. So whatever one test creates or deletes, the next test starts with an empty database again.

We can improve our test: it creates a new stamp with the factory, and checks that it is counted. The factory also creates the user and the event the stamp belongs to:

it('knows how many stamps there are in total', function () {
expect(Stamp::totalNumber())->toBe(0);
Stamp::factory()->create();
expect(Stamp::totalNumber())->toBe(1);
});

When we run this test in the terminal now, it is red:

FAILED Tests\Unit\Models\StampTest > it knows how many stamps there are in total
BadMethodCallException
Call to undefined method App\Models\Stamp::totalNumber()

Now it’s time to edit the stamp model app/Models/Stamp.php:

class Stamp extends Model
{
public function user(): BelongsTo
{
return $this->belongsTo(User::class);
}
public function event(): BelongsTo
{
return $this->belongsTo(Event::class);
}
// define the static method Stamp::totalNumber()
public static function totalNumber(): int
{
return Stamp::count();
}
}

Now the unit test is green, there is no need to refactor, and we are done on the level of unit tests.

Now we go back up to the feature test level, where the test is still red. To display the result of our method, we still have to call the method in the controller, hand the value to the view, and then display the value in the view:

use App\Models\Stamp;
use Illuminate\View\View;
class PageController extends Controller
{
public function admin(): View
{
$numberOfStamps = Stamp::totalNumber();
return view('admin', compact('numberOfStamps'));
}
}
{{-- file resources/views/admin.blade.php --}}
<x-layout title="Admin Dashboard">
<h1>Admin Dashboard</h1>
<p>{{ $numberOfStamps }} stamps in total.</p>
</x-layout>

Now the feature test is green.

Let’s look at what happens after Test Driven Development: all feature tests pass, and all commits have been pushed to the merge request. So we have now reached step 3:

  1. after picking the issue, the developer creates a feature branch and a merge request. The merge request is set to “draft” status.
  2. then the issue is implemented (we zoomed in on this step above)
  3. when the developer thinks the implementation is done, they remove the “draft” status
  4. now another developer does a code review
  5. if changes are needed, we go back to step 2.
  6. now the feature branch can be deployed to the staging server, where it is available for manual testing.
  7. if problems are found in manual testing, we go back to step 2.
  8. if the implementation is accepted, the feature branch can be merged and deployed to production

In a large team, many feature branches are open in parallel. When you start your feature branch (called “a3” in this diagram), it is up to date with main. You start working, and add commits to your own branch:

When you are ready to merge, main might have moved on: other features were finished and merged into main, so main now has new commits:

In this case you need to rebase your branch:

Terminal window
git checkout a3
git pull origin a3 # most current version of your feature branch
git rebase main
# fix problems, run the tests, fix problems again
git push --force-with-lease origin a3 # overwrite branch with rebased branch
# work on your merge request

Rebasing rewrites your commits, so that they come after the new commits on main:

Now your branch can be merged without problems.

After the implementation, it’s a good idea to read the issue again, and make sure we covered everything it describes.

We should also look at the changes in our merge request, and make sure that we did not include unnecessary changes, like these extra empty lines:

// file app/Http/Controllers/PageController.php
class PageController extends Controller
{
public function admin(): View
{
$numberOfStamps = Stamp::totalNumber();
return view('admin', compact('numberOfStamps'));
}
}

This is easy to fix: just remove them in your local code, commit and push again.

Then we can remove the draft status of the merge request.

Now a different developer in our team does a code review. Git makes this easy: the reviewer sees just the changes we made, and does not have to search through the whole codebase.

They can add comments directly in the code:

It seems we forgot to implement the first feature test on the list: only admin users may access the dashboard. Good that the code review caught it! Now we go back to the implementation to fix this.

(This part is not shown here.)

After the successful code review, we are ready to deploy the feature branch to staging. When using dokku, this is done with git on the command line:

git push dokku-staging feature_dashboard_issue_3

In the staging environment the feature can now be tested not just by developers, but also by other team members like designers or product owners, and by the customers.

We have now reached step 5 in the issue lifecycle: in the staging environment, the implementation can be tested in the browser.

  1. Define the Issue: The problem (bug) or new requirement (feature/story) is given a clear description
  2. A developer picks up the issue
  3. It is implemented (we zoomed in on this step above)
  4. It is prepared for manual testing (we zoomed in on this step above)
  5. Now the implementation can be tested in the browser, and compared to the issue
  6. If the implementation is accepted, the issue is closed, otherwise it’s back to step 2

If the implementation is accepted as a good solution to the issue, it can be deployed to production.

This is a simple version of an iterative software development process.

When you work on an existing codebase, in a project that regularly deploys new features, you will use a variation of this process.

Each team and each company also has its own conventions and standards: for branch names, for commit messages, for how many tests a feature needs before it counts as finished, and so on.