This course is still being built. Content will change. Get updates on the mailing list.

Send newsletter signup confirmation emails

You'll learn
  • Processing messages from the job queue.
  • Sending out newsletter confirmation emails through Postmark.

We've now arrived at the place where we can start sending out some emails.

As promised, we will do this by processing the messages coming into our job queue, and then writing a job to send out the newsletter confirmation emails. Let's start with the emailer.

The Emailer

We'll create another new component in our app, called the Emailer. Its responsibility will be to send HTML and text emails through Postmark.

Why Postmark, and not Amazon SES? Because in my experience, Postmark makes sending emails nice and simple, and emails get through spam filters more. They support marketing and transactional emails, have good support, are quite inexpensive, and their HTTP API is simple to use. So go ahead and setup an account with them. For now, you only need to create an account and add a so-called sender signature, because we'll be using their test environment called the sandbox.

If you don't like this choice of Postmark, feel free to substitute the email sending code in the Emailer with your own. There's only a small part you need to change.

Bonus: Transactional and marketing emails?

Like always, follow along or get the code with:

$ git fetch && git checkout --track golangdk/newsletter-email

See the diff on Github.

Let's get to it. Open up messaging/emailer.go and add:

messaging/emailer.go
package messaging import ( "bytes" "context" "embed" "encoding/json" "fmt" "io" "net/http" "strings" "time" "go.uber.org/zap" "canvas/model" ) const ( marketingMessageStream = "broadcast" transactionalMessageStream = "outbound" ) // nameAndEmail combo, of the form "Name <email@example.com>" type nameAndEmail = string //go:embed emails var emails embed.FS // Emailer can send transactional and marketing emails through Postmark. // See https://postmarkapp.com/developer type Emailer struct { baseURL string client *http.Client log *zap.Logger marketingFrom nameAndEmail token string transactionalFrom nameAndEmail } type NewEmailerOptions struct { BaseURL string Log *zap.Logger MarketingEmailAddress string MarketingEmailName string Token string TransactionalEmailAddress string TransactionalEmailName string } func NewEmailer(opts NewEmailerOptions) *Emailer { return &Emailer{ baseURL: opts.BaseURL, client: &http.Client{Timeout: 3 * time.Second}, log: opts.Log, marketingFrom: createNameAndEmail(opts.MarketingEmailName, opts.MarketingEmailAddress), token: opts.Token, transactionalFrom: createNameAndEmail(opts.TransactionalEmailName, opts.TransactionalEmailAddress), } } // SendNewsletterConfirmationEmail with a confirmation link. // This is a transactional email, because it's a response to a user action. func (e *Emailer) SendNewsletterConfirmationEmail(ctx context.Context, to model.Email, token string) error { keywords := map[string]string{ "base_url": e.baseURL, "action_url": e.baseURL + "/newsletter/confirm?token=" + token, } return e.send(ctx, requestBody{ MessageStream: transactionalMessageStream, From: e.transactionalFrom, To: to.String(), Subject: "Confirm your subscription to the Canvas newsletter", HtmlBody: getEmail("confirmation_email.html", keywords), TextBody: getEmail("confirmation_email.txt", keywords), }) } // requestBody used in Emailer.send. // See https://postmarkapp.com/developer/user-guide/send-email-with-api type requestBody struct { MessageStream string From nameAndEmail To nameAndEmail Subject string HtmlBody string TextBody string } // send using the Postmark API. func (e *Emailer) send(ctx context.Context, body requestBody) error { bodyAsBytes, err := json.Marshal(body) if err != nil { return fmt.Errorf("error marshalling request body to json: %w", err) } request, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.postmarkapp.com/email", bytes.NewReader(bodyAsBytes)) if err != nil { return fmt.Errorf("error creating request: %w", err) } request.Header.Set("Accept", "application/json") request.Header.Set("Content-Type", "application/json") request.Header.Set("X-Postmark-Server-Token", e.token) response, err := e.client.Do(request) if err != nil { return fmt.Errorf("error making request: %w", err) } defer func() { _ = response.Body.Close() }() bodyAsBytes, err = io.ReadAll(response.Body) if err != nil { return fmt.Errorf("error reading response body: %w", err) } if response.StatusCode > 299 { e.log.Info("Error sending email", zap.Int("status", response.StatusCode), zap.String("response", string(bodyAsBytes))) return fmt.Errorf("error sending email, got status %v", response.StatusCode) } return nil } // createNameAndEmail returns a name and email string ready for inserting into From and To fields. func createNameAndEmail(name, email string) nameAndEmail { return fmt.Sprintf("%v <%v>", name, email) } // getEmail from the given path, panicking on errors. // It also replaces keywords given in the map. func getEmail(path string, keywords map[string]string) string { email, err := emails.ReadFile("emails/" + path) if err != nil { panic(err) } emailString := string(email) for keyword, replacement := range keywords { emailString = strings.ReplaceAll(emailString, "{{"+keyword+"}}", replacement) } return emailString }

There's a lot going on here, so let's step through it.

The NewEmailer function

NewEmailer is a constructor function like we've seen many times before. But looking at it and its options, we can learn a lot about the Emailer.

We need a token for the Postmark API, which we will pass to the app with environment variables. That's just like the database credentials, so nothing really new here.

The two fields called marketingFrom and transactionalFrom are just strings in a special format, that get put in the email's From header. This is the name and email address the receiver will see, and they should be different for transactional emails and marketing emails.

The client is the default HTTP client we will use to call the Postmark API. It differs from http.DefaultClient by having a request timeout of three seconds. Generally, we want timeouts on all outgoing requests from our app, which is why we create one of our own.

Last, the baseURL is the external URL for our app: the one hitting our load balancer in production. In development, it's the same as the host and port the server listens on. We need this because the email receiver will be clicking a link to confirm the email address.

SendNewsletterConfirmationEmail

The Emailer.SendNewsletterConfirmationEmail function is the one that actually sends our newsletter confirmation email. It takes the recipient to and our confirmation token and constructs a request body we will be turning into JSON and sending to the Postmark API.

This is also the function that decides which email content is being sent. We generally want to send emails as both HTML and plain text, so that the thousands of different email readers in the world can pick the one they can use.

So why not use gomponents for this, like we do in the views that are shown in the browser? Because HTML for emails is generally stuck in the 90's, and gomponents isn't. πŸ˜„ For maximum compatibility, they're written in HTML4, and layouts and styling are done with table components and inline CSS. There are some excellent free email templates available (also from Postmark) that we will use with the emailer.

Notice how we're using embed from the standard library to pull all email templates into the binary. Because we're using Docker containers, we could have added the emails directory to the container instead. But this way, we don't have to fix our Dockerfile, and the email templates are always available in memory for sending.

We're using simple string replacements for adding the clickable confirmation link to the email in the getEmail helper function, and that's really all there is to it.

The send function

I won't go through this function in detail, because it's not really that interesting. It makes a request to the Postmark API with the right headers and a JSON request body, and does a lot of error checking. If you're interested in how this works, check out the API documentation.

The email templates

As I mentioned, the emails are sent in HTML and plain text form, so we need some templates in HTML and plain text. For now, just copy these to their respective locations without reading them. You'll be seeing the contents shortly.

messaging/emails/confirmation_email.txt
Confirm your subscription to the Canvas newsletter by clicking the link below: {{action_url}} Canvas Some Street Earth
messaging/emails/confirmation_email.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="x-apple-disable-message-reformatting" /> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <meta name="color-scheme" content="light dark" /> <meta name="supported-color-schemes" content="light dark" /> <title></title> <style type="text/css" rel="stylesheet" media="all"> /* Base ------------------------------ */ body { width: 100% !important; height: 100%; margin: 0; -webkit-text-size-adjust: none; } a { color: #3869D4; } a img { border: none; } td { word-break: break-word; } .preheader { display: none !important; visibility: hidden; mso-hide: all; font-size: 1px; line-height: 1px; max-height: 0; max-width: 0; opacity: 0; overflow: hidden; } /* Type ------------------------------ */ body, td, th { font-family: "Nunito Sans", Helvetica, Arial, sans-serif; } h1 { margin-top: 0; color: #333333; font-size: 22px; font-weight: bold; text-align: left; } h2 { margin-top: 0; color: #333333; font-size: 16px; font-weight: bold; text-align: left; } h3 { margin-top: 0; color: #333333; font-size: 14px; font-weight: bold; text-align: left; } td, th { font-size: 16px; } p, ul, ol, blockquote { margin: .4em 0 1.1875em; font-size: 16px; line-height: 1.625; } p.sub { font-size: 13px; } /* Utilities ------------------------------ */ .align-right { text-align: right; } .align-left { text-align: left; } .align-center { text-align: center; } /* Buttons ------------------------------ */ .button { background-color: #3869D4; border-top: 10px solid #3869D4; border-right: 18px solid #3869D4; border-bottom: 10px solid #3869D4; border-left: 18px solid #3869D4; display: inline-block; color: #FFF; text-decoration: none; border-radius: 3px; box-shadow: 0 2px 3px rgba(0, 0, 0, 0.16); -webkit-text-size-adjust: none; box-sizing: border-box; } @media only screen and (max-width: 500px) { .button { width: 100% !important; text-align: center !important; } } body { background-color: #FFF; color: #333; } p { color: #333; } .email-wrapper { width: 100%; margin: 0; padding: 0; -premailer-width: 100%; -premailer-cellpadding: 0; -premailer-cellspacing: 0; } .email-content { width: 100%; margin: 0; padding: 0; -premailer-width: 100%; -premailer-cellpadding: 0; -premailer-cellspacing: 0; } /* Masthead ----------------------- */ .email-masthead { padding: 25px 0; text-align: center; } .email-masthead_name { font-size: 16px; font-weight: bold; color: #A8AAAF; text-decoration: none; text-shadow: 0 1px 0 white; } /* Body ------------------------------ */ .email-body { width: 100%; margin: 0; padding: 0; -premailer-width: 100%; -premailer-cellpadding: 0; -premailer-cellspacing: 0; } .email-body_inner { width: 570px; margin: 0 auto; padding: 0; -premailer-width: 570px; -premailer-cellpadding: 0; -premailer-cellspacing: 0; } .email-footer { width: 570px; margin: 0 auto; padding: 0; -premailer-width: 570px; -premailer-cellpadding: 0; -premailer-cellspacing: 0; text-align: center; } .email-footer p { color: #A8AAAF; } .body-action { width: 100%; margin: 30px auto; padding: 0; -premailer-width: 100%; -premailer-cellpadding: 0; -premailer-cellspacing: 0; text-align: center; } .body-sub { margin-top: 25px; padding-top: 25px; border-top: 1px solid #EAEAEC; } .content-cell { padding: 35px; } /*Media Queries ------------------------------ */ @media only screen and (max-width: 600px) { .email-body_inner, .email-footer { width: 100% !important; } } @media (prefers-color-scheme: dark) { body { background-color: #333333 !important; color: #FFF !important; } p, ul, ol, blockquote, h1, h2, h3, span { color: #FFF !important; } .email-masthead_name { text-shadow: none !important; } } :root { color-scheme: light dark; supported-color-schemes: light dark; } </style> <!--[if mso]> <style type="text/css"> .f-fallback { font-family: Arial, sans-serif; } </style> <![endif]--> </head> <body> <span class="preheader">Confirm your subscription to the Canvas newsletter.</span> <table class="email-wrapper" width="100%" cellpadding="0" cellspacing="0" role="presentation"> <tr> <td align="center"> <table class="email-content" width="100%" cellpadding="0" cellspacing="0" role="presentation"> <tr> <td class="email-masthead"> <a href="{{base_url}}" class="f-fallback email-masthead_name"> Canvas </a> </td> </tr> <!-- Email Body --> <tr> <td class="email-body" width="570" cellpadding="0" cellspacing="0"> <table class="email-body_inner" align="center" width="570" cellpadding="0" cellspacing="0" role="presentation"> <!-- Body content --> <tr> <td class="content-cell"> <div class="f-fallback"> <h1>Hey!</h1> <p>Confirm your subscription to the Canvas newsletter by clicking the button below:</p> <!-- Action --> <table class="body-action" align="center" width="100%" cellpadding="0" cellspacing="0" role="presentation"> <tr> <td align="center"> <!-- Border based button https://litmus.com/blog/a-guide-to-bulletproof-buttons-in-email-design --> <table width="100%" border="0" cellspacing="0" cellpadding="0" role="presentation"> <tr> <td align="center"> <a href="{{action_url}}" class="f-fallback button" target="_blank">Confirm subscription</a> </td> </tr> </table> </td> </tr> </table> <!-- Sub copy --> <table class="body-sub" role="presentation"> <tr> <td> <p class="f-fallback sub">If you’re having trouble with the button above, copy and paste the URL below into your web browser.</p> <p class="f-fallback sub">{{action_url}}</p> </td> </tr> </table> </div> </td> </tr> </table> </td> </tr> <tr> <td> <table class="email-footer" align="center" width="570" cellpadding="0" cellspacing="0" role="presentation"> <tr> <td class="content-cell" align="center"> <p class="f-fallback sub align-center"> Canvas <br>Some Street <br>Earth </p> </td> </tr> </table> </td> </tr> </table> </td> </tr> </table> </body> </html>

What, no tests?

Sometimes I don't write tests for my components. There, I said it. πŸ˜‰

The Emailer is one of those pieces of code where, in my opinion, I don't think the tradeoff between effort and gain is worth it for testing. We could make the endpoint for the Emailer configurable, spin up a local HTTP server and check that the expected headers and request body content is sent to it. But we would not be testing that we actually use the Postmark API correctly.

We could of course go one step further and make an integration test setup that hits the real Postmark API with some test tokens, but I don't like tests that depend on external services. Then you couldn't run your tests if Postmark is ever down, or you're just offline. Given that the code just makes a fancy HTTP POST request, and probably won't have to be changed a lot, I'll leave it at that.

The job runner

Now that we have the email sending code in place, we move on to the part that uses it.

Remember how the email signup handler sent a message to a queue? We will now write the component that gets this message from the queue and runs the job named in it. We will call this the job Runner. Our first job will of course be the one that sends out a newsletter signup confirmation email.

First up is the Runner itself. It does a lot of things, so have a read and we'll go through it after:

jobs/runner.go
// Package jobs has a Runner that can run registered jobs in parallel. package jobs import ( "context" "sync" "time" "go.uber.org/zap" "canvas/messaging" "canvas/model" ) // Runner runs jobs. type Runner struct { emailer *messaging.Emailer jobs map[string]Func log *zap.Logger queue *messaging.Queue } type NewRunnerOptions struct { Emailer *messaging.Emailer Log *zap.Logger Queue *messaging.Queue } func NewRunner(opts NewRunnerOptions) *Runner { if opts.Log == nil { opts.Log = zap.NewNop() } return &Runner{ emailer: opts.Emailer, jobs: map[string]Func{}, log: opts.Log, queue: opts.Queue, } } // Func is the actual work to do in a job. // The given context is the root context of the runner, which may be cancelled. type Func = func(context.Context, model.Message) error // Start the Runner, blocking until the given context is cancelled. func (r *Runner) Start(ctx context.Context) { r.log.Info("Starting") r.registerJobs() var wg sync.WaitGroup for { select { case <-ctx.Done(): r.log.Info("Stopping") wg.Wait() return default: r.receiveAndRun(ctx, &wg) } } } // receiveAndRun jobs. func (r *Runner) receiveAndRun(ctx context.Context, wg *sync.WaitGroup) { m, receiptID, err := r.queue.Receive(ctx) if err != nil { r.log.Info("Error receiving message", zap.Error(err)) // Sleep a bit to not hammer the queue if there's an error with it time.Sleep(time.Second) return } // If there was no message there is nothing to do if m == nil { return } name, ok := (*m)["job"] if !ok { r.log.Info("Error getting job name from message") return } job, ok := r.jobs[name] if !ok { r.log.Info("No job with this name", zap.String("name", name)) return } wg.Add(1) go func() { defer wg.Done() log := r.log.With(zap.String("name", name)) defer func() { if rec := recover(); rec != nil { log.Info("Recovered from panic in job", zap.Any("recover", rec)) } }() before := time.Now() if err := job(ctx, *m); err != nil { log.Info("Error running job", zap.Error(err)) return } after := time.Now() duration := after.Sub(before) log.Info("Successfully ran job", zap.Duration("duration", duration)) // We use context.Background as the parent context instead of the existing ctx, because if we've come // this far we don't want the deletion to be cancelled. deleteCtx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() if err := r.queue.Delete(deleteCtx, receiptID); err != nil { log.Info("Error deleting message, job will be repeated", zap.Error(err)) } }() } // registry provides a way to Register jobs by name. type registry interface { Register(name string, fn Func) } // Register implements registry. func (r *Runner) Register(name string, j Func) { r.jobs[name] = j }

The Runner can be summarized like this: It gets a message from the queue, and passes the message to the job that is named in the message. It runs the job in a separate goroutine, and takes care of error handling and some logging. It finally deletes the message from the queue only if the job run is successful. It does all this in a loop until the context given to Start is cancelled. After the cancel, it ways for any running jobs to finish before returning.

Setup

The setup code in NewRunner takes the Queue and the Emailer through dependency injection. It also sets up an empty jobs map, which is used to map job names to a jobs.Func.

The jobs.Func is an alias for a function that takes a context.Context and a model.Message from the queue, and returns an error. This is the definition of a job.

The receive loop

In Start, we call registerJobs as the first thing. It's very similar to setupRoutes from the Server, and we'll get back to it in a minute.

Then, in a loop we check if the context given to Start is cancelled. In that case, we wait for remaining jobs to finish, using a sync.WaitGroup, and break out of the loop after.

In receiveAndRun, we start by getting the message from the queue, blocking for a maximum of 20 seconds. This is because we set the queue wait time to 20 seconds earlier, so we don't need to do any request throttling. If a message comes in, we receive and start processing it immediately.

After a few error checks, we call wg.Add(1) on the sync.WaitGroup mentioned previously. Importantly, this is done just before the goroutine is started. Otherwise, we could have a race condition where wg.Wait is called and returns before the goroutine has been run, and then we wouldn't be waiting for it!

Right after, so we don't forget, we call defer wg.Done(). This keeps the Add and Done close together in code. It's a nice pattern to follow with these kinds of wait groups and also mutexes and locks in general.

Because this part of the code runs in a separate goroutine, we set up a panic handler, using the built-in function recover. Nothing outside the goroutine can catch a panic from inside the goroutine, which is why we do it this way. Otherwise, a panic in one of our jobs would make the whole app crash, which is usually not what we want.

The job itself is run with the context given to the runner itself. That way, jobs can know if the context has been cancelled (for example, if the app must be restarted), and can react accordingly. Otherwise, jobs set their own deadlines.

A job also receives the raw message from the queue. That way, the runner doesn't have to know anything about what's inside the message, apart from the job name.

The last (and very important) thing we need to do is delete the message from the queue. Remember from earlier, that if a message is not deleted from a queue within a configurable time frame, it will be redelivered. This would mean running the job again, which we would only like to do in failure cases.

Setting up jobs

The registerJobs function mentioned goes into jobs/register.go:

jobs/register.go
package jobs func (r *Runner) registerJobs() { }

I've added it to a separate file, because just like server/routes.go, we need to change the function every time we add a new job, and it's nice and easy to find here.

Let's test the Runner

Before we proceed with defining the email sending job, let's make sure that the Runner actually does what it's supposed to. We'll create an integration test that uses a real queue, registers a test job, sends a message to the queue, and checks that the job is run successfully. Open up jobs/runner_test.go and add:

jobs/runner_test.go
package jobs_test import ( "context" "testing" "github.com/matryer/is" "go.uber.org/zap" "go.uber.org/zap/zapcore" "go.uber.org/zap/zaptest/observer" "canvas/integrationtest" "canvas/jobs" "canvas/model" ) type testRegistry map[string]jobs.Func func (r testRegistry) Register(name string, fn jobs.Func) { r[name] = fn } func TestRunner_Start(t *testing.T) { integrationtest.SkipIfShort(t) t.Run("starts the runner and runs jobs until the context is cancelled", func(t *testing.T) { is := is.New(t) queue, cleanup := integrationtest.CreateQueue() defer cleanup() log, logs := newLogger() runner := jobs.NewRunner(jobs.NewRunnerOptions{ Log: log, Queue: queue, }) ctx, cancel := context.WithCancel(context.Background()) runner.Register("test", func(ctx context.Context, m model.Message) error { foo, ok := m["foo"] is.True(ok) is.Equal("bar", foo) cancel() return nil }) err := queue.Send(context.Background(), model.Message{"job": "test", "foo": "bar"}) is.NoErr(err) // This blocks until the context is cancelled by the job function runner.Start(ctx) is.Equal(3, logs.Len()) is.Equal("Starting", logs.All()[0].Message) is.Equal("Successfully ran job", logs.All()[1].Message) is.Equal("Stopping", logs.All()[2].Message) }) } func newLogger() (*zap.Logger, *observer.ObservedLogs) { core, logs := observer.New(zapcore.InfoLevel) return zap.New(core), logs }

Because the component we're testing uses goroutines to run the jobs, we need to be sure that the test waits for the job to finish, and doesn't just finish and never check the assertions inside the job function. We do this with a little trick: the job itself cancels the Runner's context. Because Runner.Start blocks until the context is cancelled, the test automatically waits for the job function to complete and cancels the context. Nice.

If you run the test now, you'll get an error, because our Queue.Receive function returns an error if the context is cancelled. Let's change that behavior so that it just returns without an error instead:

messaging/queue.go
// Package messaging is for components that enable messaging to other systems. package messaging import ( "context" "encoding/json" "strings" "sync" "time" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/service/sqs" "go.uber.org/zap" "canvas/model" ) // … // Receive a message and its receipt ID from the queue. Returns nil if no message is available. func (q *Queue) Receive(ctx context.Context) (*model.Message, string, error) { if q.url == nil { if err := q.getQueueURL(ctx); err != nil { return nil, "", err } } output, err := q.Client.ReceiveMessage(ctx, &sqs.ReceiveMessageInput{ QueueUrl: q.url, WaitTimeSeconds: int32(q.waitTime.Seconds()), }) if err != nil { if strings.Contains(err.Error(), "context canceled") { return nil, "", nil } return nil, "", err } if len(output.Messages) == 0 { return nil, "", nil } var m model.Message if err := json.Unmarshal([]byte(*output.Messages[0].Body), &m); err != nil { return nil, "", err } return &m, *output.Messages[0].ReceiptHandle, nil } // …

Let's also create a separate test for this:

messaging/queue_test.go
package messaging_test import ( "context" "testing" "github.com/matryer/is" "canvas/integrationtest" "canvas/model" ) func TestQueue(t *testing.T) { integrationtest.SkipIfShort(t) // … t.Run("receive does not return an error if the context is already cancelled", func(t *testing.T) { is := is.New(t) queue, cleanup := integrationtest.CreateQueue() defer cleanup() // Send first, to get the queue URL when the context is not cancelled err := queue.Send(context.Background(), model.Message{}) ctx, cancel := context.WithCancel(context.Background()) cancel() m, _, err := queue.Receive(ctx) is.NoErr(err) is.Equal(nil, m) }) }

If you run your tests now, everything should work as expected!

The SendNewsletterConfirmationEmail job

Now that both the Emailer and Runner are in place, the job to send out newsletter confirmation emails is really simple. This is good news, because it means that future jobs that depend on these will probably be equally simple. Future you will thank present you. πŸ˜‡

Let's have a look at the new SendNewsletterConfirmationEmail job:

jobs/email.go
package jobs import ( "context" "errors" "fmt" "time" "canvas/model" ) type newsletterconfirmationEmailSender interface { SendNewsletterConfirmationEmail(ctx context.Context, to model.Email, token string) error } // SendNewsletterConfirmationEmail to a newsletter subscriber. func SendNewsletterConfirmationEmail(r registry, es newsletterconfirmationEmailSender) { r.Register("confirmation_email", func(ctx context.Context, m model.Message) error { ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) defer cancel() to, ok := m["email"] if !ok { return errors.New("no email address in message") } token, ok := m["token"] if !ok { return errors.New("no token in message") } if err := es.SendNewsletterConfirmationEmail(ctx, model.Email(to), token); err != nil { return fmt.Errorf("error sending newsletter confirmation email: %w", err) } return nil }) }

So what's going on here?

First of all, this style of having a package-level function that takes dependencies and registers itself somewhere could look slightly familiar to you. As already mentioned, this is because it's the same way we register our HTTP handlers in the handlers package. The same design decisions apply here: by making the function register itself, we can test that the right job name is being used. We can also use small interfaces of just the parts we care about for the dependencies, like the newsletterconfirmationEmailSender interface here, which also makes testing easier. Note that is knows nothing about the job runner or the queue.

The job definition itself is fairly simple: we check for the email address and token, and then send the email using the Emailer. The only thing that's really interesting is the fact that we don't use the context passed to the function at all. Why is that? Why go to the trouble of passing it at all then?

To understand this, remember where the context comes from: all the way from Runner.Start. It is the context that is cancelled when the Runner should stop. But in our case, even though the Runner is supposed to stop, we would still like to finish the job of sending this email, because we know it doesn't take a long time. In fact, because the local context has a timeout of 10 seconds, this is the maximum amount of time it will take to finish this job. Probably, it will take much less time. If this job were about processing a large file that could take minutes, that would be a different matter, and we would check the passed context for cancellation.

Now we just need to register our new job:

jobs/register.go
package jobs func (r *Runner) registerJobs() { SendNewsletterConfirmationEmail(r, r.emailer) }

Testing the job

We're going to test this job using a mock of the newsletterconfirmationEmailSender interface:

jobs/email_test.go
package jobs_test import ( "context" "errors" "testing" "github.com/matryer/is" "canvas/jobs" "canvas/model" ) type mockConfirmationEmailer struct { err error to model.Email token string } func (m *mockConfirmationEmailer) SendNewsletterConfirmationEmail(ctx context.Context, to model.Email, token string) error { m.to = to m.token = token return m.err } func TestSendConfirmationEmail(t *testing.T) { r := testRegistry{} t.Run("passes the recipient email and token to the email sender", func(t *testing.T) { is := is.New(t) emailer := &mockConfirmationEmailer{} jobs.SendNewsletterConfirmationEmail(r, emailer) job, ok := r["confirmation_email"] is.True(ok) err := job(context.Background(), model.Message{"email": "you@example.com", "token": "123"}) is.NoErr(err) is.Equal("you@example.com", emailer.to.String()) is.Equal("123", emailer.token) }) t.Run("errors on email sending failure", func(t *testing.T) { is := is.New(t) emailer := &mockConfirmationEmailer{err: errors.New("wire is cut")} jobs.SendNewsletterConfirmationEmail(r, emailer) job := r["confirmation_email"] err := job(context.Background(), model.Message{"email": "you@example.com", "token": "123"}) is.True(err != nil) }) }

There are two tests: one to check the happy path of sending the email with the right data, and one for when the emailer errors. Note how we check the error case by passing an error to the mockConfirmationEmailer.

The final glue

Because we have two new components, the Emailer and the Runner, we need to start them up in our main start function:

cmd/server/main.go
// Package main is the entry point to the server. It reads configuration, sets up logging and error handling, // handles signals from the OS, and starts and stops the server. package main import ( "context" "fmt" "os" "os/signal" "syscall" "time" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/sqs" "github.com/aws/smithy-go/logging" "github.com/maragudk/env" "go.uber.org/zap" "golang.org/x/sync/errgroup" "canvas/jobs" "canvas/messaging" "canvas/server" "canvas/storage" ) // release is set through the linker at build time, generally from a git sha. // Used for logging and error reporting. var release string func main() { os.Exit(start()) } func start() int { _ = env.Load() logEnv := env.GetStringOrDefault("LOG_ENV", "development") log, err := createLogger(logEnv) if err != nil { fmt.Println("Error setting up the logger:", err) return 1 } log = log.With(zap.String("release", release)) defer func() { // If we cannot sync, there's probably something wrong with outputting logs, // so we probably cannot write using fmt.Println either. So just ignore the error. _ = log.Sync() }() host := env.GetStringOrDefault("HOST", "localhost") port := env.GetIntOrDefault("PORT", 8080) awsConfig, err := config.LoadDefaultConfig(context.Background(), config.WithLogger(createAWSLogAdapter(log)), config.WithEndpointResolver(createAWSEndpointResolver()), ) if err != nil { log.Info("Error creating AWS config", zap.Error(err)) return 1 } queue := createQueue(log, awsConfig) s := server.New(server.Options{ Database: createDatabase(log), Host: host, Log: log, Port: port, Queue: queue, }) r := jobs.NewRunner(jobs.NewRunnerOptions{ Emailer: createEmailer(log, host, port), Log: log, Queue: queue, }) var eg errgroup.Group ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT) defer stop() eg.Go(func() error { <-ctx.Done() if err := s.Stop(); err != nil { log.Info("Error stopping server", zap.Error(err)) return err } return nil }) eg.Go(func() error { r.Start(ctx) return nil }) if err := s.Start(); err != nil { log.Info("Error starting server", zap.Error(err)) return 1 } if err := eg.Wait(); err != nil { return 1 } return 0 } // … func createEmailer(log *zap.Logger, host string, port int) *messaging.Emailer { return messaging.NewEmailer(messaging.NewEmailerOptions{ BaseURL: env.GetStringOrDefault("BASE_URL", fmt.Sprintf("http://%v:%v", host, port)), Log: log, MarketingEmailName: env.GetStringOrDefault("MARKETING_EMAIL_NAME", "Canvas bot"), MarketingEmailAddress: env.GetStringOrDefault("MARKETING_EMAIL_ADDRESS", "bot@marketing.example.com"), Token: env.GetStringOrDefault("POSTMARK_TOKEN", ""), TransactionalEmailName: env.GetStringOrDefault("TRANSACTIONAL_EMAIL_NAME", "Canvas bot"), TransactionalEmailAddress: env.GetStringOrDefault("TRANSACTIONAL_EMAIL_ADDRESS", "bot@transactional.example.com"), }) }

To be able to run this in your development setup, you need to do a few things in Postmark:

  1. Make sure you've registered a sender signature. If you just want to code along and try out email sending, registering one to use for both transactional and marketing emails is fine. Then you can set both MARKETING_EMAIL_ADDRESS and TRANSACTIONAL_EMAIL_ADDRESS to the same email address in your .env file.
  2. Set up a new server in sandbox mode. When creating a new server, your screen should look something like this:

    Screenshot of the new server dialogue box at Postmark.
  3. Grab the server API token from the API Tokens tab and put it into your .env file under the name POSTMARK_TOKEN.

Then it's time to try things out. Start your app, head over to localhost:8080, and enter any valid email address. No real email will be sent because of the sandbox mode your Postmark server was set up with. You should see the thank you view in your browser, and something like this in your app logs:

2021-11-27T05:17:25Z    INFO    jobs/runner.go:110      Successfully ran job    {"release": "", "name": "confirmation_email", "duration": "833.974125ms"}

Note the job duration: we just saved ourselves from waiting for the HTTP handler for almost a second! This is the power of running jobs asynchronously to your HTTP handlers. Your app will work at lightning speed. ⚑️

Now you can check your Postmark account for the email, if you click the Default Transactional Stream under your server and go to the Activity tab. The screen looks something like this:

Screenshot of the sandbox of a server in Postmark.

Feel free to click around in the Postmark interface. You can see the email you sent in both HTML and plain text form, view metadata, and more.

What's next?

The components we have now are a powerful thing. You can quickly create new jobs and tell them to run in your HTTP handlers, using the Queue and Runner behind the scenes. All the infrastructure is in place, so adding new jobs is as easy as writing the job code, registering the job with a name, and sending a message to the queue from somewhere with that job name.

As you might have guessed, this is exactly what we need to do next. Because when you click the link in the confirmation email, you're presented with a good old 404 page. So next up: creating a handler that checks the token and confirms the email in the database, and sends out a welcome email. All without adding new infrastructure components!

If you've made it this far again, well done. Understanding and writing these building blocks can feel tedious, but it's well worth it when you can evolve your app quickly using them. Feel free to celebrate now. It's okay to dance in front of your computer. πŸ’»πŸ‘―

Review questions

Sign up or log in to get review questions by email! πŸ“§

Questions?

Get help on Twitter or by email.