Teamwork Leads to Active Refreshing Sheets

Smartsheet Engineering has fostered a culture where the customer experience is king, and every member of the team takes pride in making sure that Smartsheet products are a delight to use. 

These delightful features are built through the collaboration of multiple teams. Teams from development, design, testing, and product management all play a role in preparing features for our customers. And feedback from every one of those teams is vital for the production of quality features.

Such was the case when Imran Khawaja, Lead Software Development Engineer on the Automation team, and Djordje Savovic, Sr. Quality Assurance Engineer, worked together to complete a task, only to realize that they needed to pivot in order to build a truly useful solution.

Imran's team is responsible for writing rules that automatically fire when events happen in a sheet, and one of the recent rules his team was asked to write would lock a row when certain changes in the row occured. The team checked in a solution, but as it reached Djordje to test he pointed out that locking the row would make life difficult for other collaborators who were in the sheet at the same time.

"Unless we're active-refreshing the sheet," Djordje said, "locking the row automatically will not be particularly useful to users because the shared user can keep editing the sheet until it's refreshed."

Djordje knew about this behavior in the application code because of some research he did previously--unrelated to the current task. He was curious about the security of the Smartsheet application and learned how the sheet saving process worked.

The working relationship between the various teams in Smartsheet Engineering encourages dialog, and Djordje enjoys working at a place that values his input.

"That's the benefit of this company," Djordje said. "Nobody discards your opinion. They always take it into consideration."

Knowing how the sheet saving process worked came in handy for Djordje when he and Imran were planning the test phase of the row lock feature. 

After Djordje pointed out how the checked-in solution would affect the shared users, Imran and the dev team brainstormed and came up with a wild idea. Why not "fake" it?

"Our crazy idea was," Imran said, "can we somehow get the latest changes to the other collaborators without them doing a real refresh?  We started looking into it, and our first idea was to do a 'fake' merge save. We'll call all the same logic that does a merge save, but we won't actually save to the database and just send a response back to the client.

"Because our app has been around for 10-12 years, the saving process stuff all works. There's a lot of logic that goes into it to make sure that we get all the latest information and don't throw away anything that the other user just saved."

The Automation team didn't come to this decision lightly, nor was it their first idea. Their first idea was to compare two states of the sheet and essentially perform a diff. 

"This approach proved to be naive because of the way that we do merging of things," Imran said.

So Imran and his team decided to start small, and then take an incremental approach at solving the problem. 

"We went with the safe approach," Imran said. "We're processing the changes to existing cells on existing rows. Whenever we see those changes occur in a sheet we send those changes back up to the other shared users.

"We're going with the most safe thing that we know--which is just changes to existing cells in existing rows -  and then build upon that. That's the strategy, so that we can be sure that the system will scale."

Not only did the team start small with which events would fire an automatic update, but they also limited the number of active users on a sheet to five. So, as long as there are five or less users looking at a sheet at the same time, and changes are made to existing cells, the sheet will actively refresh for everyone. If there are more users looking at a sheet, then everyone will receive the toaster messages that notify of changes--which is how the application behaved before the active refresh feature.

"We came up with that number by running a query to find what are the most active sheets and  how many users were there in a specified period of time," Imran explained. "There was 15--so I set the limit at 1/3 of that. If the performance on the system is good, we'll up that number."

So, if there are fewer than five collaborators working on a sheet at a given time, here are the order of events for an active refresh to occur. 

User A and User B are both working in a sheet. User A makes a change, and then their data is saved. At this point a message is sent by the Collab Service on the server through a web socket to User B's client. The message delivers the latest modified timestamp of the sheet. 

User B then compares the timestamp in the message with its own timestamp. If User B is out of date, it checks the server to see if it can active refresh. The active refresh process first checks to make sure that the changes to the sheet were only in existing cells in existing rows. 

"If that's the case," Imran said, "then it creates the same JavaScript package we have now, and then sends it back to the user."

If, for some reason, the sheet can not active refresh, the event and reason is logged for the development teams to analyze and prioritize which events to make available next.

Smartsheet engineers understand that to make their software stable and delightful to use, it requires teams to work together--considering input from all sides, continually iterating on problems to find the best solution, and building things in a way that provides a strong foundation for future feature development.