Please enable JavaScript to view the comments powered by Disqus.

Dec
18
Christmas break - the Beta is getting closer

 

North Creek is going on vacation for the next week. If I have time, I will send out some tweets and update my Facebook page, but this blog will likely be pretty quiet until the New Year.

In the new year, North Creek will be launching two products - Listmaker and CXL. Branding for these products may change as we go through Beta. A quick summary:

1) Listmaker:

This is going to be a free app that will let you collect data from your teammates / friends / meetup participants / whatever. You upload a spreadsheet with an empty table in it - no data, just column headers. We convert that table into a simple web form. We give you a URL - something like

https://northcreek.ca/listmaker/7468db85-2d8f-466d-abdf-063ffd707030 (Just a sample, not live)

and a QR code that points to the same URL.

You share those links however you like. Anyone who follows those links gets access to an empty form where they can provide the data for your spreadsheet. You can get notified when anyone or a certain number of users fill in the form.

When you come back to northcreek.ca, you log in (using an email address or some public identity (Facebook / Twitter / LinkedIn / Yahoo / Microsoft) and you get to see all the forms that you have in our system. As the editor of the form, you are the only one who gets to see the data others have provided.

When you are satisfied that you have collected enough data (you have enough for a pot-luck, or you have the whole team's contact info), you close the collection and download the spreadsheet, with all the data filled in for you.

This will be ad-supported, with options to pay to remove ads and add different styles to the default look of the form.

2) CXL:

This will be our flagship application. CXL will take fully-featured spreadsheets and create a "document" record around them. Within that document record, you will be able to assign visibility and editing rights to people based on their North Creek identity (which again, can be based on email or on another public identity provider). You will be able to track changes to the spreadsheet by row, column, formula, format, or data.

CXL will give you a preview of the data and the charts in your spreadsheet, as of the latest version, or of any version in the document history. You will be able to make minor edits to the spreadsheet from the web interface, but primarily, the web page will serve as a window into the repository that surrounds the spreadsheet. If someone wants to "fork" your spreadsheet, they can do so and retain the document history up to the fork.

CXL will be compatible with .xls and .xlsx records to start, with other file formats becoming available in the future.

The goal will be to create an understanding of a spreadsheet and to secure it as a useful, reliable part of your business process. To that end, North Creek will work on integrating other data sources into your document so that you can create a template and fill it with data using web services or other cloud data providers. The outputs will always include a real spreadsheet document, but other outputs could be created as well.

In future releases, CXL could extract a template out of your spreadsheet, removing volatile data. You could also upload a template and associate it with a document record. The outcome of that could be the creation of a community resource of spreadsheet templates, with the ability to share useful spreadsheet designs with your team, your company, or the world.

I think spreadsheets are under-utilized because IT can't get a handle on them. CXL will allow you to see and manage your spreadsheet inventory, removing IT's concerns and solving your problems at the same time.

CXL will start with basic functionality in January. Listmaker will launch when ready. Stay tuned!

Contact: Email, Twitter, Facebook, LinkedIn, or this blog.

Categories: Business , Software

Dec
17
Single Point of Failure

 

Single Point of Failure...

Dreaded words, but a fact of life for many small companies.

Truck Number:

How many employees being hit by a truck on the way to work would it take to shut your company down?

If the number is 1, you are pretty vulnerable, and that employee should be asking for a raise. If that person is the owner, well, then, you have a job, not a business.

Of course, I am "preaching to the choir!" I am bootstrapping North Creek Software, and very few people understand my vision, and fewer understand my code. Our truck number is most definitely 1.

Back to Single Point of Failure (SPOF). If employees are a SPOF, you may be limited in what you can do. If there is a technical SPOF (a server, a router, a power source) it is up to you to determine whether the price of diversifying that function can be justified, given the likelihood of failure of that SPOF.

My power went out for about 30 minutes at home today. I found other things to do, but it delayed this blog post and caused me to go through the restart process for a couple of home servers (development and test). Next month, while I'm in an active Beta, that will be a problem, but for now I can deal.

One of the benefits of moving to a SaaS product or to custom software is that things can be backed up, moved off-site, and made available via redundant servers. If you depend on your file server sitting in the corner, you have a SPOF. If you run everything through a spreadsheet, you have a SPOF if that spreadsheet gets corrupted. You have backups, but you are talking about losing work and filling in a gap.

I understand that your budget / amount of mental space you have for this problem may be limited. Moving documents to a repository solves a lot of issues. Moving to a smart repository turns things into a win-win.

North Creek is building that smart repository. Get in on the ground floor, email me, or contact me via Facebook or Twitter. My LinkedIn account also links back to this blog. Help me steer this project.

Talk to you soon.

Categories: Business

Dec
16
Productivity - and first steps to community

Productivity

Some days you can just get into the "flow" of things and stuff just gets done. Some days you have to push harder to get going. Usually, once I get started, with my tools set up and even one small thing accomplished, I can keep the momentum going and crank some real output.

I was told (it might have been in "The Pragmatic Programmer" by Hunt and Thomas) to leave one easy task undone at the end of every day, so you'd be starting on a downhill slope the next morning.

Are your tools set up for that?

If your business runs on a bunch of disconnected files and folders, and you have to remember where you are and what the next step is every morning, you may have a difficult time getting into a "flow state". On the other hand, if you are in the real world, and your tools are designed to fit some idealized view of how business works, you may be spending your time trying to force your tools to fit your reality.

I love the freedom of starting with an empty spreadsheet when I'm trying to imagine something. You can just create a table over here, a formula over there, and then a chart just because. I might be a bit of a spreadsheet geek, or maybe I haven't spent enough 50-hour weeks staring at them. I can always go back to my programming environment or the command line. But spreadsheets are only useful once you've done the setup work - setting them up is a cost.

What if someone else has a template that will get you halfway there? What if we can share solutions (not data, but formulas, chart designs, table arrangements)?

Open source software has bloomed through sites like SourceForge and GitHub. North Creek sees the potential for a smaller, but still global exchange of spreadsheet development effort. CXL is the first step to making that happen.

Nobody is going to share their corporate data with the world. But if your spreadsheet is already in the CXL repository, and it has been analyzed and compartmentalized using North Creek's software, a template could be extracted out of your working spreadsheet that could help others.

What do you gain? Membership in a community that recognizes your contribution, and access to great ideas from others. The sense that you're not the only one dealing with this mess.

First steps - you could be part of creating something new. Sign up by emailing me, "like" North Creek Software on Facebook or follow me on Twitter, or comment on the blog (within the blog using Disqus, or using the ShareThis buttons up top). Do you work with spreadsheets? Are you a whiz at something? Would it be nice to have a library of templates to share, so that you would be halfway there and going downhill when you start tomorrow morning?

Categories: Business , Software

Dec
16
Data Flows and Simple Tools

Data Flows and Simple Tools

Professional programmers look at your business as something like a chemical refinery. There are inputs (prospective customers, investment, [lots of] labour) and transformations, like chemical reactions, leading to products (physical products, or happy customers, or both) as well as waste products. It is in your best interest that the flow has as little turbulence as possible. Lean manufacturing is focused on reducing waste.

North Creek's first marketing tagline was "Let your business flow." I wanted to help you understand where data was being wasted in your business, where errors were creeping in, and how we could automate things to make your work easier.

Now I'm focused on creating a couple of products that will help you do that, without having to fly me in to walk you through the process. (You're probably a ways from Onoway, Alberta). We'll start small. If you, like most small businesses, use spreadsheets to track numbers or projects, CXL will help you by adding a layer of "adult supervision" over your spreadsheets. You'll have a history of what is changing. You'll have the ability to go back in time, or to branch a useful spreadsheet into another area.

But what could happen is that you'll start to see the data flow through your company. Spreadsheets filled with numbers change like crazy. If those numbers are the real description of what's happening in your company, then it's the flow of those numbers that we want to understand and manage.

I want to help you create tools that don't change, that help you get a consistent look at the data that does change. Good spreadsheet design gives you a place in the spreadsheet where data comes in, a place where calculations are performed, and a place where you look for the result. If those are all mixed together, it will be hard to manage the flow.

But the internet is full of people smarter than me when it comes to spreadsheets. Point me in the right direction - let's help each other!

Meanwhile, CXL will open it's beta in January. Sign up by emailing me, follow @NorthCreekSoft on twitter, like North Creek Software on Facebook.

"Let your business flow!" :-)

Categories: Business , Methodology

Dec
13
Integration via Spreadsheet?

 

In many ways, businesses use Excel as an edge node in the graph of integration pathways. Data comes in to your spreadsheet, is folded, mutilated and spindled, and turned into a table or chart you can put into your slide deck.

We all know Excel and its competitors can do more than that, but for all the reasons I've described previously, few businesses choose to go that way. It's too easy to change a spreadsheet - it's one big sandbox for playing with data! So businesses lock things down with custom software or integration tools.

If we had a repository that tracked and provided "adult supervision" for spreadsheets, could we use spreadsheets closer to the middle of the integration graph? Could we trust some spreadsheets as tools to provide some visibility into the guts of your data as it moves through your business? Instead of being stuck in a database that only a DBA "Wizard" can peer into, would it be helpful to be able to see data as it is being transformed?

Excel and others are pretty good at subscribing to data feeds (using ODBC or other connectivity, or through CSV file imports). Part of North Creek's technology stack is Apache POI, which is a library that takes apart a spreadsheet like a chicken, allowing me to see its guts and play with each individual cell. CXL was envisioned as a repository with intelligence, allowing businesses to trust that their spreadsheets were not being changed and distributed without the document owner's knowledge. What if there was the ability to put data feeds into CXL, not just spreadsheets?

I am not planning on re-inventing any wheels. I would embed Mule as a front end and require some configuration on your part to set up the data feeds. But any data source can be expressed as an html page (I am using jmesa for now) or as a spreadsheet table. This could be interesting.

If you want to help steer North Creek's product vision, sign up for the beta by emailing me at dave@northcreek.ca.

You can also like my Facebook page or follow me on Twitter. If you are interested in receiving these blog posts via email, tell me in an email!

Categories: Business , Methodology , Software

Dec
12
Good Spreadsheets vs. Bad Spreadsheets

Spreadsheet "Design"

Spreadsheets are usually "grown", not designed. If you are responsible for a spreadsheet that is vital to a business process, relied upon by others - you may not be able to describe the parts of the spreadsheet. It just is. There are the places you go to update the data. There are the changes you make to the dates. There are the parts that always break and the parts that never change.

So what makes a good spreadsheet?

Let me know in the comments or on social media what you think. I have an opinion, but there are experts out there!

Spreadsheets are a container for data and programming code. So my experience as a database administrator and designer, and as a programmer across multiple languages (and eras) suggests that there may be some patterns that would be useful.

1) Separation of Concerns.

Many programmers use the shorthand SoC for this, and it is one of the pillars of good design in most coding schools of thought. Make every component of your [thing] do one thing, and that one thing well. For spreadsheets, that probably means looking at the structure of your worksheets. If you have everything on one worksheet, it might make things easier to move some of the content onto another sheet and use relative links to bring the data back for calculation purposes.

A rule of thumb - separate the spreadsheets into sections based on how fast things change. If you have parts that never change, put them in a worksheet and don't touch that worksheet. If it changes, you should know why. If you have stuff that changes all the time, isolate that from the rest of the document so that the volatility doesn't lead to accidentally overwriting some of the longer-lived stuff. If you have stuff that changes on a schedule, work on a naming scheme and put all of the date-sensitive stuff on one worksheet. Then you can turn that into a template and change only that stuff on a schedule.

2) Don't Repeat Yourself

This comes from "The Pragmatic Programmer" by Hunt and Thomas (right up there on my bookshelf). If you enter data twice, one of the times will be wrong - not every time, but often enough that there will be a problem at some time in the future. So try to avoid the cut and paste, and especially the re-typing. If you can automate the export and import of data from your other tools, do it! The fewer the steps involved, the better.

There are plenty more - let me know your best ideas, or the source of your best Spreadsheet tips!

...Follow me on Twitter @NorthCreekSoft - Like North Creek Software on Facebook - email me at dave@northcreek.ca...

Categories: Methodology , Software

Dec
11
Spreadsheets and Databases

 

Custom Spreadsheets.

It's obvious to the database administrator when he (usually he - ask me sometime about one awesome female DBA I worked with) hears those words that this is going to be a mess.

Someone who doesn't know much about computers (or they wouldn't have done this on a spreadsheet) has poured hours and hours into a set of worksheets that work "just right." All the formulas are just so, the graphs are exactly what we want to see, and the tables can be sorted and filtered to a fare-thee-well.

Until it doesn't work.

Someone adds a row in the wrong place. Data is improperly typed in. The wrong format data is imported. Someone forgets to change the "Month" cell that a bunch of careful analysis is based on.

Suddenly, it's IT's problem. Or - if you are a small business owner - it's your problem, and even if you created the spreadsheet, it's not an easy solution.

Where's the backup? There used to be formulas in this column - what were they supposed to be? How was this supposed to be updated at the end of the month?

If North Creek is going to solve the "Spreadsheet Problem," we need to hear all of these stories. The gnarly, painful, exhausting work of untangling a spreadsheet that fit like my old pair of slippers, until suddenly it didn't. Let me know in the comments. Like "North Creek Software" on Facebook. Follow @NorthCreekSoft on twitter. Find me (David Block) on LinkedIn. Whatever - I want to hear your horror stories.

I also want to connect with you experts that know the tools to get us out of this mess. I will not have the answer to every problem - despite my optimistic marketing claims. I am going to solve one big piece of it.

How is that going to work?

You have to think in multiple dimensions - I'm going to start with 5(!) and we'll see how far we get.

1) Spreadsheet Data
Spreadsheets are tables, and we love putting data in table form. All SQL databases work off of tables. CXL will extract the data from the spreadsheet you upload, so that we can track changes to it as the document evolves.

2) Spreadsheet Programming
The difference between a Word table and an Excel data table is that in Excel, you can do things to the data! You can use formulas (formulae if you are pedantic) to do calculations. You can use VBA to do things even Excel can't do. You can then display the results in informative graphs and charts. Everything in the document is stored in the file (xml if you are using .xlsx), so it can all be extracted and stored. CXL will isolate the programming parts of the document as much as possible, so that we know what's editable in what way. Some times (model generation) the data is static and the calculations are volatile. Usually, the calculations change relatively slowly, and the data underneath changes.

3) Cell formatting
Cells are little documents, and each has a format. Many times, the formats will be consistent down a column, or across a row. But the differences are meaningful. Spreadsheets are meant to have a displayed component somewhere. That needs to be tracked. This is where .csv export and import fail.

4) Rows
One of the simple pleasures of working with a spreadsheet is the moment when you realize you've forgotten a row, and you just insert it in the right place in the middle of things. As long as it doesn't break that absolute reference you put in that other place... Rows are often inserted or deleted, and tracking that change by individual cell (a) is tedious, and (b) misses a piece of information. The user was inserting or deleting a data record of some kind (usually), or adding a calculation that wasn't there before. This could be noticed as a change.

5) Columns
Of course, anything you can do to rows, you can do to columns. Again, it may be a whole new record, or more likely, a whole new attribute across all the records, that you are adding or removing. New calculations across the entire data set. Storing that change as a bunch of cell additions turns that useful information into a bunch of noise.

CXL is aiming to be a smart, spreadsheet-savvy document repository. Sharepoint and other document management systems can store versioned spreadsheets and keep the "latest" copy available. CXL will break your spreadsheet down across at least these five dimensions, tracking each change where it makes the most sense. The evolution of your document becomes a story CXL tells back to you. Of course there will be security and user roles and easy access. But CXL will see into your document, helping you develop it and protecting you from that change that necessitates that call to some expensive "expert" who will tell you to quit using spreadsheets and get a real application already.

I think we can make your spreadsheet into a part of your business that you don't have to worry about.

dave@northcreek.ca 

P.S. The Beta will start in January. We are looking for a few more businesses who have spreadsheet problems that need fixing. Let me know if you are one of them!

Categories: Business , Software

Dec
10
Spreadsheets and the failure of IT

 

When I was a corporate programmer, spreadsheets were evil.

There were too many of them, they contained important and valuable data, and we (the IT people) couldn't keep them under control.

People would have them on USB sticks, on Dropbox, on their kid's laptop. And the calculations were stuck in an obfuscated binary format, hard to get at and hard to reuse.

So why are spreadsheets still around?

1) They are easy. Nothing to a good model or a good project breakdown in 20 minutes. No other tool gives you that much flexibility.

2) They are flexible. Add a column. Add a row. Change the formula on the third linked worksheet and look at the graph change.

3) They are personal. You can do things on your desktop (or laptop) (or tablet) that others don't have to see or laugh at you about. You can spitball a scenario. You can what-if. You can write one version of the project summary that is not for publication, then actually erase it and write a not-so-candid version.

So where do spreadsheets live?

Wherever change is happening.

A new company doesn't have the budget or the insight to commit to a database schema that won't change. It can't know whether this tool or that tool will match its process. So spreadsheets work during growth.

During disruption, trying to fit the new reality into an ERP package built for the last decade (and designed during the decade before that) leads to frustration and error. A spreadsheet tracks the data as things change around you.

CXL will use a gentle touch. Keep your spreadsheet. Keep your formulas. We'll find the data in the spreadsheet. We'll track changes as they happen, whether they are within a cell, a new row, a new column. The history will be there. And you will be able to manage change.

CXL will do branching. Spreadsheets are personal, and the other person's template may be a good starting point for them, but not for you. Or maybe you want to take a model in an entirely different direction. Branch the document, don't lose the history, and continue on your own path.

CXL will be easy. Just a file upload, and you have a document in a secure repository. There will be a public URL if you want to share with the world. There will be private access tokens that you can hand out. There will be the option to share only with a certain group of people, that have to log in using their email address. And for each level, you can restrict to preview only, download privileges, the right to propose changes, or document ownership. You can make things complicated if they need to be. But simple if that's all that you need as well.

Suggestions? Comments? Let me know. Email me at dave@northcreek.ca  to join the Beta in January, or to subscribe to the North Creek mailing list.

Categories: Methodology , Software

Dec
09
Listmaker

Listmaker will be North Creek's first product- a simple way to collect information from your friends and co-workers. Just upload an empty table with column headers. We'll create a simple web form and give you a link and a QR code. Publicize however you like, and wait for the submissions to come in. Then come back and download your filled-out spreadsheet.

Simple, effective, easy.

Categories:

Dec
09
CXL

When I worked for GNF as a Java Developer, we regularly analyzed data from experiments scanning the entire Human Genome. Our experiments were expensive to run, so we often had only two runs of the experiment. Statistical analysis with a population of 2 is impossible, so all our analysis was ad hoc and based on feel and the experience of the researchers.

The researchers that we served inevitably wanted to "take a look at the numbers" themselves. One of the major features of our software package was a "freeze-dried" compressed image of a CSV file that could be downloaded and opened in Excel. The researchers all had their own favourite Excel tricks that they would use to analyze the results of their experiments.

Fast forward to working at an enterprise software company, as an Implementation Technical Consultant. I was the senior technical resource on a team that was responsible for a Fortune 500 company's installation of our software. This was a complicated process. There were multiple tracks, overlapping deadlines, dependencies, and conflicts. We used Basecamp to manage our communication with our customers. What we really ended up doing was trading spreadsheets with tables of TODO lists and deadlines. Basecamp became a simple document repository. We wasted time and effort figuring out Basecamp's versioning system and bumping up against its storage maximums. Attempts at migrating to Google Docs ran up against the limits of multiple people messing with one document.

I have had multiple conversations with people who work with spreadsheets, and all of them have a versioning problem. There are workarounds, but CXL will be a simple web-based solution that tracks your spreadsheet, allows you to integrate changes from teammates, control access, and keep control.

Sign up for the Beta in January 2015 by emailing dave@northcreek.ca.

Categories:

Dec
09
North Creek Software

Christmas is coming - get in on the BETA by emailing dave@northcreek.ca before Christmas. More details to follow...

Categories:

Dec
06
#HoldMeToIt #EMyth and social media

Hello again - back on this blog platform. I will be centralizing North Creek's messaging going forward - but this blog is still the best place to read what I'm thinking.

EMyth wants to raise their profile over the New Year, and I'm happy to help them out - "The E-Myth Revisited" is one of the formative books of this entrepeneurial journey, and I look forward to following through on the business design that follows from the discovery of product-market fit (from Steve Blank).

So, in January 2015, North Creek Software will launch our Beta, listen to our testers, and iterate until we have something that is really valuable to today's dynamic workplace. #HoldMeToIt.

2015 is the year that I quit relying on my wife to pay the bills and start giving her holidays! #HoldMeToIt.

Thanks for making me make my goals explicit, #EMyth!

Categories: Books , Business