North Creek is going on vacation for the next week. If I have time, I
will send out some tweets and update my Facebook page, but this blog
will likely be pretty quiet until the New Year.
In the new year, North Creek will be launching two products - Listmaker
and CXL. Branding for these products may change as we go through Beta. A
quick summary:
1) Listmaker:
This is going to be a free app that will let you collect data from your
teammates / friends / meetup participants / whatever. You upload a
spreadsheet with an empty table in it - no data, just column headers. We
convert that table into a simple web form. We give you a URL - something
like
https://northcreek.ca/listmaker/7468db85-2d8f-466d-abdf-063ffd707030
(Just a sample, not live)
and a QR code that points to the same URL.
You share those links however you like. Anyone who follows those links
gets access to an empty form where they can provide the data for your
spreadsheet. You can get notified when anyone or a certain number of
users fill in the form.
When you come back to northcreek.ca, you log in (using an email address
or some public identity (Facebook / Twitter / LinkedIn / Yahoo /
Microsoft) and you get to see all the forms that you have in our system.
As the editor of the form, you are the only one who gets to see the data
others have provided.
When you are satisfied that you have collected enough data (you have
enough for a pot-luck, or you have the whole team's contact info), you
close the collection and download the spreadsheet, with all the data
filled in for you.
This will be ad-supported, with options to pay to remove ads and add
different styles to the default look of the form.
2) CXL:
This will be our flagship application. CXL will take fully-featured
spreadsheets and create a "document" record around them. Within that
document record, you will be able to assign visibility and editing
rights to people based on their North Creek identity (which again, can
be based on email or on another public identity provider). You will be
able to track changes to the spreadsheet by row, column, formula,
format, or data.
CXL will give you a preview of the data and the charts in your
spreadsheet, as of the latest version, or of any version in the document
history. You will be able to make minor edits to the spreadsheet from
the web interface, but primarily, the web page will serve as a window
into the repository that surrounds the spreadsheet. If someone wants to
"fork" your spreadsheet, they can do so and retain the document history
up to the fork.
CXL will be compatible with .xls and .xlsx records to start, with other
file formats becoming available in the future.
The goal will be to create an understanding of a spreadsheet and to
secure it as a useful, reliable part of your business process. To that
end, North Creek will work on integrating other data sources into your
document so that you can create a template and fill it with data using
web services or other cloud data providers. The outputs will always
include a real spreadsheet document, but other outputs could be created
as well.
In future releases, CXL could extract a template out of your
spreadsheet, removing volatile data. You could also upload a template
and associate it with a document record. The outcome of that could be
the creation of a community resource of spreadsheet templates, with the
ability to share useful spreadsheet designs with your team, your
company, or the world.
I think spreadsheets are under-utilized because IT can't get a handle on
them. CXL will allow you to see and manage your spreadsheet inventory,
removing IT's concerns and solving your problems at the same time.
CXL will start with basic functionality in January. Listmaker will
launch when ready. Stay tuned!
Contact: Email, Twitter,
Facebook,
LinkedIn,
or this blog.
Categories:
Business
,
Software
Christmas break - the Beta is getting closer
Single Point of Failure...
Dreaded words, but a fact of life for many small companies.
Truck Number:
How many employees being hit by a truck on the way to work would it take
to shut your company down?
If the number is 1, you are pretty vulnerable, and that employee should
be asking for a raise. If that person is the owner, well, then, you have
a job, not a business.
Of course, I am "preaching to the choir!" I am bootstrapping North Creek
Software, and very few people understand my vision, and fewer understand
my code. Our truck number is most definitely 1.
Back to Single Point of Failure (SPOF). If employees are a SPOF, you may
be limited in what you can do. If there is a technical SPOF (a server, a
router, a power source) it is up to you to determine whether the price
of diversifying that function can be justified, given the likelihood of
failure of that SPOF.
My power went out for about 30 minutes at home today. I found other
things to do, but it delayed this blog post and caused me to go through
the restart process for a couple of home servers (development and test).
Next month, while I'm in an active Beta, that will be a problem, but for
now I can deal.
One of the benefits of moving to a SaaS product or to custom software is
that things can be backed up, moved off-site, and made available via
redundant servers. If you depend on your file server sitting in the
corner, you have a SPOF. If you run everything through a spreadsheet,
you have a SPOF if that spreadsheet gets corrupted. You have backups,
but you are talking about losing work and filling in a gap.
I understand that your budget / amount of mental space you have for this
problem may be limited. Moving documents to a repository solves a lot of
issues. Moving to a smart repository turns things into a win-win.
North Creek is building that smart repository. Get in on the ground
floor, email me, or contact me
via Facebook
or Twitter.
My LinkedIn
account also links back to this blog. Help me steer this project.
Talk to you soon.
Categories:
Business
Single Point of Failure
Productivity
Some days you can just get into the "flow" of things and stuff just gets
done. Some days you have to push harder to get going. Usually, once I
get started, with my tools set up and even one small thing accomplished,
I can keep the momentum going and crank some real output.
I was told (it might have been in "The Pragmatic Programmer" by Hunt and
Thomas) to leave one easy task undone at the end of every day, so you'd
be starting on a downhill slope the next morning.
Are your tools set up for that?
If your business runs on a bunch of disconnected files and folders, and
you have to remember where you are and what the next step is every
morning, you may have a difficult time getting into a "flow state". On
the other hand, if you are in the real world, and your tools are
designed to fit some idealized view of how business works, you may be
spending your time trying to force your tools to fit your reality.
I love the freedom of starting with an empty spreadsheet when I'm trying
to imagine something. You can just create a table over here, a formula
over there, and then a chart just because. I might be a bit of a
spreadsheet geek, or maybe I haven't spent enough 50-hour weeks staring
at them. I can always go back to my programming environment or the
command line. But spreadsheets are only useful once you've done the
setup work - setting them up is a cost.
What if someone else has a template that will get you halfway there?
What if we can share solutions (not data, but formulas, chart designs,
table arrangements)?
Open source software has bloomed through sites like SourceForge and
GitHub. North Creek sees the potential for a smaller, but still global
exchange of spreadsheet development effort. CXL is the first step to
making that happen.
Nobody is going to share their corporate data with the world. But if
your spreadsheet is already in the CXL repository, and it has been
analyzed and compartmentalized using North Creek's software, a template
could be extracted out of your working spreadsheet that could help
others.
What do you gain? Membership in a community that recognizes your
contribution, and access to great ideas from others. The sense that
you're not the only one dealing with this mess.
First steps - you could be part of creating something new. Sign up by emailing
me, "like" North Creek Software on Facebook
or follow me on Twitter,
or comment on the blog (within the blog using Disqus, or using the
ShareThis buttons up top). Do you work with spreadsheets? Are you a whiz
at something? Would it be nice to have a library of templates to share,
so that you would be halfway there and going downhill when you start
tomorrow morning?
Categories:
Business
,
Software
Productivity - and first steps to community
Data Flows and Simple Tools
Professional programmers look at your business as something like a
chemical refinery. There are inputs (prospective customers, investment,
[lots of] labour) and transformations, like chemical reactions, leading
to products (physical products, or happy customers, or both) as well as
waste products. It is in your best interest that the flow has as little
turbulence as possible. Lean manufacturing is focused on reducing waste.
North Creek's first marketing tagline was "Let your business flow." I
wanted to help you understand where data was being wasted in your
business, where errors were creeping in, and how we could automate
things to make your work easier.
Now I'm focused on creating a couple of products that will help you do
that, without having to fly me in to walk you through the process.
(You're probably a ways from Onoway,
Alberta). We'll start small. If you, like most small businesses, use
spreadsheets to track numbers or projects, CXL will help you by adding a
layer of "adult supervision" over your spreadsheets. You'll have a
history of what is changing. You'll have the ability to go back in time,
or to branch a useful spreadsheet into another area.
But what could happen is that you'll start to see the data flow through
your company. Spreadsheets filled with numbers change like crazy. If
those numbers are the real description of what's happening in your
company, then it's the flow of those numbers that we want to understand
and manage.
I want to help you create tools that don't change, that help you get a
consistent look at the data that does change. Good spreadsheet
design gives you a place in the spreadsheet where data comes in, a
place where calculations are performed, and a place where you look for
the result. If those are all mixed together, it will be hard to manage
the flow.
But the internet is full of people smarter than me when it comes to
spreadsheets. Point me in the right direction - let's help each other!
Meanwhile, CXL will open it's beta in January. Sign up by emailing
me, follow @NorthCreekSoft
on twitter, like North
Creek Software on Facebook.
"Let your business flow!" :-)
Categories:
Business
,
Methodology
Data Flows and Simple Tools
In many ways, businesses use Excel as an edge node in the graph of
integration pathways. Data comes in to your spreadsheet, is folded,
mutilated and spindled, and turned into a table or chart you can put
into your slide deck.
We all know Excel and its competitors can do more than that, but for all
the reasons I've described previously, few businesses choose to go that
way. It's too easy to change a spreadsheet - it's one big sandbox for
playing with data! So businesses lock things down with custom software
or integration tools.
If we had a repository that tracked and provided "adult supervision" for
spreadsheets, could we use spreadsheets closer to the middle of the
integration graph? Could we trust some spreadsheets as tools to provide
some visibility into the guts of your data as it moves through your
business? Instead of being stuck in a database that only a DBA "Wizard"
can peer into, would it be helpful to be able to see data as it is being
transformed?
Excel and others are pretty good at subscribing to data feeds (using
ODBC or other connectivity, or through CSV file imports). Part of North
Creek's technology stack is Apache POI,
which is a library that takes apart a spreadsheet like a chicken,
allowing me to see its guts and play with each individual cell. CXL was
envisioned as a repository with intelligence, allowing businesses to
trust that their spreadsheets were not being changed and distributed
without the document owner's knowledge. What if there was the ability to
put data feeds into CXL, not just spreadsheets?
I am not planning on re-inventing any wheels. I would embed Mule
as a front end and require some configuration on your part to set up the
data feeds. But any data source can be expressed as an html page (I am
using jmesa for now) or
as a spreadsheet table. This could be interesting.
If you want to help steer North Creek's product vision, sign up for the
beta by emailing me at dave@northcreek.ca.
You can also like my Facebook
page or follow me on Twitter.
If you are interested in receiving these blog posts via email, tell me
in an email!
Categories:
Business
,
Methodology
,
Software
Integration via Spreadsheet?
Spreadsheet "Design"
Spreadsheets are usually "grown", not designed. If you are responsible
for a spreadsheet that is vital to a business process, relied upon by
others - you may not be able to describe the parts of the spreadsheet.
It just is. There are the places you go to update the data. There are
the changes you make to the dates. There are the parts that always break
and the parts that never change.
So what makes a good spreadsheet?
Let me know in the comments or on social media what you think. I have an
opinion, but there are experts out there!
Spreadsheets are a container for data and programming code. So my
experience as a database administrator and designer, and as a programmer
across multiple languages (and eras) suggests that there may be some
patterns that would be useful.
1) Separation of Concerns.
Many programmers use the shorthand SoC for this, and it is one of the
pillars of good design in most coding schools of thought. Make every
component of your [thing] do one thing, and that one thing well. For
spreadsheets, that probably means looking at the structure of your
worksheets. If you have everything on one worksheet, it might make
things easier to move some of the content onto another sheet and use
relative links to bring the data back for calculation purposes.
A rule of thumb - separate the spreadsheets into sections based on how
fast things change. If you have parts that never change, put them in a
worksheet and don't touch that worksheet. If it changes, you should know
why. If you have stuff that changes all the time, isolate that from the
rest of the document so that the volatility doesn't lead to accidentally
overwriting some of the longer-lived stuff. If you have stuff that
changes on a schedule, work on a naming scheme and put all of the
date-sensitive stuff on one worksheet. Then you can turn that into a
template and change only that stuff on a schedule.
2) Don't Repeat Yourself
This comes from "The Pragmatic Programmer" by Hunt and Thomas (right up
there on my bookshelf). If you enter data twice, one of the times will
be wrong - not every time, but often enough that there will be a problem
at some time in the future. So try to avoid the cut and paste, and
especially the re-typing. If you can automate the export and import of
data from your other tools, do it! The fewer the steps involved, the
better.
There are plenty more - let me know your best ideas, or the source of
your best Spreadsheet tips!
...Follow me on Twitter @NorthCreekSoft
- Like North Creek
Software on Facebook - email me at dave@northcreek.ca...
Categories:
Methodology
,
Software
Good Spreadsheets vs. Bad Spreadsheets
Custom Spreadsheets.
It's obvious to the database administrator when he (usually he - ask me
sometime about one awesome female DBA I worked with) hears those words
that this is going to be a mess.
Someone who doesn't know much about computers (or they wouldn't have
done this on a spreadsheet) has poured hours and hours into a set of
worksheets that work "just right." All the formulas are just so, the
graphs are exactly what we want to see, and the tables can be sorted and
filtered to a fare-thee-well.
Until it doesn't work.
Someone adds a row in the wrong place. Data is improperly typed in. The
wrong format data is imported. Someone forgets to change the "Month"
cell that a bunch of careful analysis is based on.
Suddenly, it's IT's problem. Or - if you are a small business owner -
it's your problem, and even if you created the spreadsheet, it's not an
easy solution.
Where's the backup? There used to be formulas in this column - what were
they supposed to be? How was this supposed to be updated at the end of
the month?
If North Creek is going to solve the "Spreadsheet Problem," we need to
hear all of these stories. The gnarly, painful, exhausting work of
untangling a spreadsheet that fit like my old pair of slippers, until
suddenly it didn't. Let me know in the comments. Like "North Creek
Software" on Facebook. Follow @NorthCreekSoft on twitter. Find me (David
Block) on LinkedIn. Whatever - I want to hear your horror stories.
I also want to connect with you experts that know the tools to get us
out of this mess. I will not have the answer to every problem - despite
my optimistic marketing claims. I am going to solve one big piece of it.
How is that going to work?
You have to think in multiple dimensions - I'm going to start with 5(!)
and we'll see how far we get.
1) Spreadsheet Data
Spreadsheets are tables, and we love putting data
in table form. All SQL databases work off of tables. CXL will extract
the data from the spreadsheet you upload, so that we can track changes
to it as the document evolves.
2) Spreadsheet Programming
The difference between a Word table and an
Excel data table is that in Excel, you can do things to the data! You
can use formulas (formulae if you are pedantic) to do calculations. You
can use VBA to do things even Excel can't do. You can then display the
results in informative graphs and charts. Everything in the document is
stored in the file (xml if you are using .xlsx), so it can all be
extracted and stored. CXL will isolate the programming parts of the
document as much as possible, so that we know what's editable in what
way. Some times (model generation) the data is static and the
calculations are volatile. Usually, the calculations change relatively
slowly, and the data underneath changes.
3) Cell formatting
Cells are little documents, and each has a format.
Many times, the formats will be consistent down a column, or across a
row. But the differences are meaningful. Spreadsheets are meant to have
a displayed component somewhere. That needs to be tracked. This is where
.csv export and import fail.
4) Rows
One of the simple pleasures of working with a spreadsheet is
the moment when you realize you've forgotten a row, and you just insert
it in the right place in the middle of things. As long as it doesn't
break that absolute reference you put in that other place... Rows are
often inserted or deleted, and tracking that change by individual cell
(a) is tedious, and (b) misses a piece of information. The user was
inserting or deleting a data record of some kind (usually), or adding a
calculation that wasn't there before. This could be noticed as a change.
5) Columns
Of course, anything you can do to rows, you can do to
columns. Again, it may be a whole new record, or more likely, a whole
new attribute across all the records, that you are adding or removing.
New calculations across the entire data set. Storing that change as a
bunch of cell additions turns that useful information into a bunch of
noise.
CXL is aiming to be a smart, spreadsheet-savvy document repository.
Sharepoint and other document management systems can store versioned
spreadsheets and keep the "latest" copy available. CXL will break your
spreadsheet down across at least these five dimensions, tracking each
change where it makes the most sense. The evolution of your document
becomes a story CXL tells back to you. Of course there will be security
and user roles and easy access. But CXL will see into your document,
helping you develop it and protecting you from that change that
necessitates that call to some expensive "expert" who will tell you to
quit using spreadsheets and get a real application already.
I think we can make your spreadsheet into a part of your business that
you don't have to worry about.
dave@northcreek.ca
P.S. The Beta will start in January. We are looking for a few more
businesses who have spreadsheet problems that need fixing. Let me know
if you are one of them!
Categories:
Business
,
Software
Spreadsheets and Databases
When I was a corporate programmer, spreadsheets were evil.
There were too many of them, they contained important and valuable data,
and we (the IT people) couldn't keep them under control.
People would have them on USB sticks, on Dropbox, on their kid's laptop.
And the calculations were stuck in an obfuscated binary format, hard to
get at and hard to reuse.
So why are spreadsheets still around?
1) They are easy. Nothing to a good model or a good project breakdown in
20 minutes. No other tool gives you that much flexibility.
2) They are flexible. Add a column. Add a row. Change the formula on the
third linked worksheet and look at the graph change.
3) They are personal. You can do things on your desktop (or laptop) (or
tablet) that others don't have to see or laugh at you about. You can
spitball a scenario. You can what-if. You can write one version of the
project summary that is not for publication, then actually erase it and
write a not-so-candid version.
So where do spreadsheets live?
Wherever change is happening.
A new company doesn't have the budget or the insight to commit to a
database schema that won't change. It can't know whether this tool or
that tool will match its process. So spreadsheets work during growth.
During disruption, trying to fit the new reality into an ERP package
built for the last decade (and designed during the decade before that)
leads to frustration and error. A spreadsheet tracks the data as things
change around you.
CXL will use a gentle touch. Keep your spreadsheet. Keep your formulas.
We'll find the data in the spreadsheet. We'll track changes as they
happen, whether they are within a cell, a new row, a new column. The
history will be there. And you will be able to manage change.
CXL will do branching. Spreadsheets are personal, and the other person's
template may be a good starting point for them, but not for you. Or
maybe you want to take a model in an entirely different direction.
Branch the document, don't lose the history, and continue on your own
path.
CXL will be easy. Just a file upload, and you have a document in a
secure repository. There will be a public URL if you want to share with
the world. There will be private access tokens that you can hand out.
There will be the option to share only with a certain group of people,
that have to log in using their email address. And for each level, you
can restrict to preview only, download privileges, the right to propose
changes, or document ownership. You can make things complicated if they
need to be. But simple if that's all that you need as well.
Suggestions? Comments? Let me know. Email me at dave@northcreek.ca
to join the Beta in January, or to subscribe to the North Creek mailing
list.
Categories:
Methodology
,
Software
Spreadsheets and the failure of IT
Listmaker will be North Creek's first product- a simple way to collect
information from your friends and co-workers. Just upload an empty table
with column headers. We'll create a simple web form and give you a link
and a QR code. Publicize however you like, and wait for the submissions
to come in. Then come back and download your filled-out spreadsheet.
Simple, effective, easy.
Categories:
Listmaker
Dec
09 CXL
When I worked for GNF as a
Java Developer, we regularly analyzed data from experiments scanning the
entire Human Genome. Our experiments were expensive to run, so we often
had only two runs of the experiment. Statistical analysis with a
population of 2 is impossible, so all our analysis was ad hoc and based
on feel and the experience of the researchers.
The researchers that we served inevitably wanted to "take a look at the
numbers" themselves. One of the major features of our software package
was a "freeze-dried" compressed image of a CSV file that could be
downloaded and opened in Excel. The researchers all had their own
favourite Excel tricks that they would use to analyze the results of
their experiments.
Fast forward to working at an enterprise software company, as an
Implementation Technical Consultant. I was the senior technical resource
on a team that was responsible for a Fortune 500 company's installation
of our software. This was a complicated process. There were multiple
tracks, overlapping deadlines, dependencies, and conflicts. We used Basecamp to
manage our communication with our customers. What we really ended up
doing was trading spreadsheets with tables of TODO lists and deadlines.
Basecamp became a simple document repository. We wasted time and effort
figuring out Basecamp's versioning system and bumping up against its
storage maximums. Attempts at migrating to Google Docs ran up against
the limits of multiple people messing with one document.
I have had multiple conversations with people who work with
spreadsheets, and all of them have a versioning problem. There are
workarounds, but CXL will be a simple web-based solution that tracks
your spreadsheet, allows you to integrate changes from teammates,
control access, and keep control.
Sign up for the Beta in January 2015 by emailing dave@northcreek.ca.
Categories:
CXL
Christmas is coming - get in on the BETA by emailing
dave@northcreek.ca
before Christmas. More details to follow...
Categories:
North Creek Software
Hello again - back on this blog platform. I will be centralizing North
Creek's messaging going forward - but this blog is still the best place
to read what I'm thinking.
EMyth wants to raise their profile over the New Year, and I'm happy to
help them out - "The E-Myth Revisited" is one of the formative books of
this entrepeneurial journey, and I look forward to following through on
the business design that follows from the discovery of product-market
fit (from Steve Blank).
So, in January 2015, North Creek Software will launch our Beta, listen
to our testers, and iterate until we have something that is really
valuable to today's dynamic workplace. #HoldMeToIt.
2015 is the year that I quit relying on my wife to pay the bills and
start giving her holidays! #HoldMeToIt.
Thanks for making me make my goals explicit, #EMyth!
Categories:
Books
,
Business
#HoldMeToIt #EMyth and social media
