Go and protobuf quick start

For a long long time I’ve wanted to use a format other than JSON for sending messages between two points. But I’ve not yet had the opportunity to use anything other than JSON for work, and haven’t written enough personal stuff to run into it yet. So I’ve decided to create an arbitrary program just to try it out protobuf!

Later I might try out some other protocols like Thrift or MessagePack.

Anyway let’s do this!

So full disclosure, there’s already a really good guide out there, though it is using protobuf 2 instead of 3. It really helped me when getting started with these, so I recommend checking it out.

Anyway, first you have to install protobuf. If you’re on a mac and use homebrew, it’s fairly approachable. brew install protobuf will get you the what you need. If you’re on another OS, you should check out the project’s install section.

After you have protobuf, you’ll need to get the proto project, and the protoc-gen-go, to generate code.

go get -u github.com/golang/protobuf/proto
go get -u github.com/golang/protobuf/protoc-gen-go

The above should do it. Though you could also use modules if you want.

Now let’s write our message

And keep it simple. How about some text and a number, and we’ll call it “message”. That’ll make our message.proto file look like this:

// This needs to be first so we can opt in to using protobuf version 3.
syntax = "proto3";

// This is next and will set the package name on our generated code.
// By default it will use the name of the .proto file.
// In this case the default would be "package message"
package main;

// This is our actual message.
message Message {
  string text = 1;
  int64 number = 2;
}

I’ve added comments to be helpful, but here’s a link to the actual protobuf documentation if you want some more info.

Generate the code!

Since we’ve installed everything, we can protoc --go_out=. *.proto in the directory with our message.proto file. This should generate a file called message.pb.go. Towards the top of that file, we can see something like our message definition:

type Message struct {
	Text                 string   `protobuf:"bytes,1,opt,name=type,proto3" json:"type,omitempty"`
	Number               int64    `protobuf:"varint,2,opt,name=number,proto3" json:"number,omitempty"`
	XXX_NoUnkeyedLiteral struct{} `json:"-"`
	XXX_unrecognized     []byte   `json:"-"`
	XXX_sizecache        int32    `json:"-"`
}

It’s got some extra fields I know nothing about, but it certainly looks like our text and number.

Now we should create a simple program to create a message, Marshal it, and then Unmarshal that. Then we can say we’ve used a protobuf! That program looks like this:

package main

import (
	"fmt"
	"log"

	"github.com/golang/protobuf/proto"
)

/* Remember, Message looks something like this:
type Message struct {
	Text   string
	Number int
}
*/

func main() {
	message := &Message{
		Text:   "text",
		Number: 1,
	}

	m, err := proto.Marshal(message)
	if err != nil {
		log.Fatalf("Marshalling error: %s\n", err)
	}

	newMessage := &Message{}
	err = proto.Unmarshal(m, newMessage)
	if err != nil {
		log.Fatalf("Unmarshalling error: %s\n", err)
	}

	if message.Text != newMessage.Text || message.Number != newMessage.Number {
		log.Fatalln("Oh no! Our messages did not match.")
	}

	fmt.Printf("Successfully unmarshalled: %+v\n", newMessage)
}

When I run it with go run simple_message.go message.pb.go, I get the output Successfully unmarshalled: text:"text" number:1.

Aaaand, we’re done!

In case you want to see it, all the code can be found here.

Hopefully this or the guide I linked to earlier can help you get up and running with protobuf. And hopefully I’ll be playing with some other protocols soon!