May 2, 2020

Keep Time Explicit

In a lot of software is operations are depending on time. For example creating invoices, statistics, showing the latest news so on so forth. In this blog post, I use an example summing up bought items in the last week. We might have multiple layers in our system, like database access, places to do extra processing, and a web interface. I’m using C# here, but it applies to most mainstream languages. As you can see, to get the right database records the cure uses the current time.

Example Database Access:
public class Purchases
{
    public IList<Purchase> LastWeeksPurchases()
    {
        // Assume more complex queries, let's say joins with users, applied coupons, discounts at the time and god knows what
        return Database.InTransaction(conn => conn.Query<Purchase>(@"select * from purchases
            where dateTime < current_timestamp
            and (current_timestamp + INTERVAL '-7 day') < dateTime
            order by dateTime").ToList());
    }
}
Example Business Logic:
public class PurchaseSummaries
{
    private readonly Purchases _purchases;

    // ...

    public PurchaseSummary SummarizeLastWeek()
    {
        // Assume more complex logic. Maybe it goes to different data sources, does more calculations etc
        var purchases = _purchases.LastWeeksPurchases();

        var total = purchases.Sum(e => e.Price);
        var count = purchases.Count();
        var priciest = purchases.OrderByDescending(p=>p.Price).FirstOrDefault();
        return new PurchaseSummary(total, count, priciest);
    }
}
Example Web Api:
[ApiController]
[Route("[controller]")]
public class PurchasesController : ControllerBase
{
    private readonly PurchaseSummaries _summaries;

    // ...

    [HttpGet]
    public PurchaseSummary Get()
    {
        // Assume more complex code, like access checks, showing partial or full data depending on permissions etc
        return _summaries.SummarizeLastWeek();
    }
}

Initial Testing

While you develop, you write test cases for your code. All right, no big issue. You store some test data and check if the report makes sense. The test is not hard to write and we are happy. It might look something like this:

First test:

[Test]
public void LastWeeksSummary()
{
    // Test data
    Database.InTransaction(con =>
    {
        con.Execute("truncate table purchases");
        con.Execute(
            "Insert into purchases (item, price, dateTime) values (@item, @price, @dateTime);",
            new []
            {
                new {item="older item", price=12.23m, dateTime=DateTime.UtcNow.AddDays(-30)},
                new {item="3 days old item", price=0.99m, dateTime=DateTime.UtcNow.AddDays(-3)},
                new {item="yesterday item", price=9.99m, dateTime=DateTime.UtcNow.AddDays(-1)},
                new {item="today item", price=49.99m, dateTime=DateTime.UtcNow.AddHours(-1)}
            });
    });
    var toTest = new PurchaseSummaries(new Purchases());

    var summary = toTest.SummarizeLastWeek();

    Assert.AreEqual(3, summary.Count);
    Assert.AreEqual(49.99m, summary.MostExpensiveItem.Price);
}

More Complex Scenario, Testing Pain

End of Year Report
Figure 1. Pretend End of the Year

Later we get new requirements. Let’s say at the end of a year the week only lasts till December 30th. When the new year starts only the days of the new year count. While you start writing the test you encounter the challenge that you need to set your time to the end of the year. How do you do that? Change the time of your computer would be crazy =). If you drank the Java-Spring- / ASP.NET-/ Other- dependency-injection cool-aid (examples 1, 2), you might Google around and find the recommendation to have a 'Clock' interface and get time from it. The java.time library even has a Clock interface for that. Stop right there. Using a Clock interface should be your last resort. In most cases there is a simpler and more flexible approach: Pass the time explicitly as a parameter. In our example we add a parameter for at what time we want for the summary:

Database Access:
public IList<Purchase> LastWeeksPurchases(DateTime at)
{
    // Assume more complex queries, let's say joins with applied coupons, discounts at the time and god knows what
    return Database.InTransaction(conn => conn.Query<Purchase>(@"select * from purchases
        where dateTime < @at
        and (@at + INTERVAL '-7 day') < dateTime
        order by dateTime", new {at=at}).ToList());
}

Business Logic:

public PurchaseSummary SummarizeLastWeek(DateTime at)
{
    // Assume more complex logic. Maybe it goes to different data sources, does more calculations etc
    var purchases = _purchases.LastWeeksPurchases(at);

    var total = purchases.Sum(e => e.Price);
    var count = purchases.Count();
    var priciest = purchases.OrderByDescending(p=>p.Price).FirstOrDefault();
    return new PurchaseSummary(total, count, priciest);
}

A small change, but with it testing time-related things become trivial. You set up your example data and specify the time for the test. This is great for edge cases like leap years, daylight saving time switches, and so on.

Testing Code.

[Test]
public void WeekEndsAtTheEndOfYear()
{
    var endOfYear = DateTime.Parse("2020-12-31T23:00:00Z");
    /* Snip: insert test data at the end of the year */

    var toTest = new PurchaseSummaries(new Purchases());

    var endOfYearSummary = toTest.SummarizeLastWeek(endOfYear);
    /* Check that end of year calculation is correct */
}

[Test]
public void WeekStartsAtBeginOfYear()
{
    var startOfYear = DateTime.Parse("2021-01-01T05:34:07Z");
    /* Snip: insert test data at the start of the year */

    var toTest = new PurchaseSummaries(new Purchases());

    var startOfYearSumary = toTest.SummarizeLastWeek(startOfYear);
    /* Check that end of year calculation is correct */
}

Making Time Consistent

Once you start passing time explicitly it will be clear and consistent over multiple calculations. I often see code that gets the current time multiple times for some calculation. Something like:

Sub Reports
Figure 2. Consistent Time
// The top level report used in the system
public Report ReportSummary()
{
    var report = SubSystemReport();
    var data = AnotherSetOfData();
    return new Report(report, data);
}

// Assume that this is in another module / far away
private Summary SubSystemReport()
{
    // Build up complex report. Note, it gets time again
    var time = DateTime.UtcNow;
    return BuildReportBasedOnTime(time);
}
// Assume that this is in another module / far away
private Summary AnotherSetOfData()
{
    // Query and get more data. Here again we get the time
    var time = DateTime.UtcNow;
    var someDataBaseOnTime = CalculateWith(time);
    var data = LatestData();

    return data;
}
// Assume that this is in another module / far away
private List<Data> LatestData()
{
    // You guess it, getting time again
    var time = DateTime.UtcNow;
    var queryResult = DatabaseQueryForLatestData(time);
    return queryResult;
}

As you see, we get the current time multiple times all over the system. It isn’t consistent with the different subreports. Each part gets a slightly different time, a millisecond or two apart. However, on a bad day, it might cross a day boundary or has a time jump because of system clock synchronization. Good luck debugging that. Furthermore, did you notice that passing a Clock instance won’t fix that issue? When you start passing the time explicitly this issue doesn’t exist:

// The top level report used in the system
public Report ReportSummary()
{
    var now = DateTime.UtcNow;
    var report = SubSystemReport(now);
    var data = AnotherSetOfData(now);
    return new Report(report, data);
}

// Assume that this is in another module / far away
private Summary SubSystemReport(DateTime time)
{
    // Build up complex report for the specified time
    return BuildReportBasedOnTime(time);
}
// Assume that this is in another module / far away
private Summary AnotherSetOfData(DateTime time)
{
    // Query and get more data as the specified time
    var someDataBaseOnTime = CalculateWith(time);
    var data = LatestData(time);

    return data;
}
// Assume that this is in another module / far away
private List<Data> LatestData(DateTime time)
{
    // Query for the specified time
    var queryResult = DatabaseQueryForLatestData(time);
    return queryResult;
}

Specifying Time at the System Boundary

We refactored our code to pass the time explicitly, and it made our testing easier. One day we receive complaints from a customer that their summary last weekend was wrong. However, this week it looks correct again. If we only could investigate what when wrong last weekend by turning the clock back? Oh wait, our code can do that, so why don’t we provide that capability at the system boundary, for example at the Web API level? Actually, that is a great feature. For example, in a billing system I worked on we could do that. We could rerun the billing system with a specified time to investigate any issue we saw in the past. In our example, we can support specifying the time in our Web API.

[HttpGet]
public PurchaseSummary Get(string atTime)
{
    // Support a optional time query parameter to show reports of the past
    DateTime time;
    if (string.IsNullOrEmpty(atTime))
        time = DateTime.UtcNow;
    else
        time = DateTime.Parse(atTime);
    // Assume more complex code, like access checks, showing partial or full data depending on permissions etc
    return _summaries.SummarizeLastWeek(time);
}

Tada, report from the past:

Different Result for Different Time

Summary

If you have code calculating results based on time then provide a parameter to specify the time explicitly. It makes testing code easier and avoids any time inconsistencies across sections or modules in your code.

One More Thought

I would also recommend one more thing about time. Think about time in your database. For example, try to only insert new records and not change existing ones. This allows you to investigate past issues, make reports about the past, and see how things got into a bad state. The business data can be precious. We keep a history of our source code in version control, why do we tend to edit business data in place? We probably want a history of your companies data

Some modern SQL databases support temporal tables, where you can query the past. Or you might even have the luxury to choose a database which handles time in a very explicit manner, like Datomic or Crux.

Tags: Scala Testing Clojure C# Java Development