A trip to QConSF

I have attended several tech conferences since I became a software engineer. This year I went to QConSF, a famous tech conference hosted by infoQ. I attended several talks, chatted with peers. It was great experience to attend a conference full of tech industry professionals.

Architecture trend

I went to several talks about architecture. Nubank, a startup bank in Brazil shared their evolvement in architecture which enabled them grow significantly from their birth in recent years. It’s an amazing story which broke my stereotype of finantial institution: critical safety requirement, dense regulation being obstacles of creativity. They did an amazing initial job to gain a rapid customer growth, but they soon face big problem of data writing throughput at scale (They use a git-commit style database called Datomic), they considered and tossed out the idea of sharding the database, they ended up shard by “unit of services” with global routing. Another challenge they have is, the git-commit style database make it hard for aggregation, they can’t quickly answer CEO’s question “How many customers do we have”. They ended up do log tailing for those transactional db rows to their data warehouse.

Slack talked about how they split a monolithic system to some microservices architecture: in the past they load the full snapshot all at once, which made their cold start (every morning, every Slack user) very slow and expensive. Now they reduced P99 load time from 40s to 4s.

Randy Shoup (who would later gave a great talk about “engineer yourself” later) talked about how StitchFix fixed the problem that they have a monolithic shared DB which cannot scale: they split them to a few core databases, build services on top of each, then serve client applications, and how they deal with join (basically build another service which caches the joined view, backed by a flow of event)and transaction (construct a state machine of atomic event, so that if rollback needed, run a work flow of “compensating events”) because they in turn becomes hard.

Personal Growth

Randy Shoup’s talk about “engineering yourself” simply amazes me, I would recommend everyone, even not work in tech industry to watch it once the video is available. Before they publicly share the video let me give some quick take aways: Have “growth mindset” rather than fixed mindset, and realize “I know I can improve”, neither “I cannot get better” nor “I am never good enough”: it’s easier to realize you can get better, but the trickier part is, people can have imposter syndrome thinking “never good enough”, and cannot realize that, which is actually very destructive. Trust, neither “distruct” or “being naive”, a good way to support people to grow, is instead of think people are inherently lazy and avoid responsibility, require extrinsic motivation (theory X), but believe people are intrinsically motivated, seek ownership, want to perform well(theory Y). Be confident, avoid “fear” or “arrogance”, and again imposter syndrome can do more harm than you think. Randy openly told his own story that he planned to give talk about same topic 2.5 years ago at QConNY, but because of imposter syndrome he cancelled that talk at that time. He also gave suggestions on how to build confidence: “Do it till you become it”, and “Find your core”. I feel lucky I am working in a company, Facebook, which is essentially supporting engineers to grow in this way.

Others

People from Netflix talked about how their deal with incidents, and they go further than Chaos Monkey to make disaster-like event to find problems. People from Tozny(a security company) gave a lot of suggestions on how to write good Crypto code, but one general take away for everyone is: Don’t blindly trust Google or Stackoverflow while coding, example: Google “how to encrypt strings in android”, top 3 results (all on Stackoverlow) are all wrong. People from Uber talked about their ML framework, and shared some case study of how they make use of ML in Uber Eats. Also people from Reddit how they maintain their culture when they grow from 30 people to 130 people now. There were also speakers from Facebook to share stories with the broader audience.

The conference itself

This conference has 3 days full of talks. At any talk time there are 6 talks and 1 or 2 other activities (peer discussion, AMA, townhall) happening concurrently so it’s impossible to attend them all. Videos will become public sometime later. They are making incremental improvements to the conference itself too. Like the “facilitated peer sharing” really enabled people to discuss topic on a particular area, and they implemented a convenient way to vote each talk you attended. I would recommend my engineer friends to go to conferences like QCon.