The depreciating value of human knowledge

Automation is just one facet on the broader spectrum of AI and machine intelligence. Yes, it's going to affect us all (it already is with the increasing emergence of intelligent agents and bots), but I think there is a far deeper issue here that - at least for the majority of people who haven't become immersed in the "AI" meme - is going largely unnoticed. That is, the very nature of human knowledge and how we understand the world. Machines are now doing things that - quite simply - we don't understand, and probably never will. 

I think most of us are familiar with the DIKW model (over-simplification if ever there was), but if you ascribe to this relationship between data, information, knowledge and wisdom, I think the top layers - knowledge and wisdom - are getting compressed by our growing dependencies on the bottom two layers - data and information. What will the DIKW model look like in 20 years time? I'm thinking a barely perceptible "K" and "W" layers!

If you think this is a rather outrageous prediction, I recommend reading this article from David Weinberger, who looks at how machines are rapidly outstripping our puny human abilities to understand them. And it seems we're quite happy with this situation, since being fairly lazy by nature, we're more than happy to let them make complex decisions for us. We just need to feed them the data - and there's plenty of that about! 

This quote from the piece probably best sums it up:

"As long as our computer models instantiated our own ideas, we could preserve the illusion that the world works the way our knowledge —and our models — do. Once computers started to make their own models, and those models surpassed our mental capacity, we lost that comforting assumption. Our machines have made obvious our epistemological limitations, and by providing a corrective, have revealed a truth about the universe. 

The world didn’t happen to be designed, by God or by coincidence, to be knowable by human brains. The nature of the world is closer to the way our network of computers and sensors represent it than how the human mind perceives it. Now that machines are acting independently, we are losing the illusion that the world just happens to be simple enough for us wee creatures to comprehend

We thought knowledge was about finding the order hidden in the chaos. We thought it was about simplifying the world. It looks like we were wrong. Knowing the world may require giving up on understanding it."

Should we be worried? I think so - do you?

Steve Dale

Organisational Knowledge in a Machine Intelligence era

KIN Winter Workshop 2016, 7^th December 2016.

According to Narrative Science, 62 per cent of organisations will be using Artificial Intelligence (AI) by 2018.

If you asked most people when they last encountered something that used artificial intelligence, they’d probably conjure up a mental image of robots, and might be hard pressed to think of something in everyday use. Machine intelligence and machine learning – the new synonyms for “artificial intelligence” – are on the rise and are going to be pervasive. Anyone using a smartphone is already using some sort of machine intelligence with Google Now’s suggestions, Siri’s voice recognition, or Windows Cortana personal assistant. We don’t call these “artificial intelligence”, because it’s a term that alarms some people and has earned some ridicule down the years. But it doesn’t matter what you call it; the ability to get computers to infer information that they aren’t directly supplied with, and to act on it, is already here.

But what does all this mean in a practical sense? Can – or should we -  rely on intelligent machines to do the heavy (physical and cognitive) lifting for us, and if so, what does the future hold for knowledge and information professionals?

The rise of the chatbot

It’s taken about 10 years, but social media has finally been accepted as a business tool, rather than just a means for people to waste time. If you look at any contemporary enterprise collaboration system, you’ll find social media features borrowed from Facebook or Twitter embedded into the functionality. Organisations have (finally) learnt that the goal of social technology within the workplace is not simply to maximise engagement or to facilitate collaboration, but rather to support work activities without getting in the way. Having said that, we still can’t extract ourselves from email as the primary tool for doing business. Email is dead, long live email!

Some progress then. But technology never stands still, and there’s more disruption on the way, led as usual by the consumer society. Early in 2016, we saw the introduction of the first wave of artificial intelligence technology in the form of chatbots and virtual assistants. This is being heralded as a new era in technology that some analysts have referred to as the “conversation interface”. It’s an interface that won’t require a screen or a mouse to use. There will be no need to click, swipe or type. This is an era when a screen for a device will be considered antiquated, and we won’t have to struggle with UX design. This interface will be completely conversational, and those conversations will be indistinguishable from the conversations we have with work colleagues, friends and family.

Virtual Assistants are personalised cross-platform devices that work with third-party services to respond instantly to users requests which could include online searching, purchasing, monitoring and controlling connected devices and facilitating professional tasks and interactions.

Will it be another 10 years before we see this technology accepted as a business tool? I think not, because the benefits are so apparent.  For example, given the choice of convenience and accessibility, would we still use email to get things done, or would we have a real-time conversation? Rather than force workers to stop what they’re doing and open a new application, chatbots and virtual assistants inject themselves into the places where people are already communicating. Instead of switching from a spreadsheet to bring up a calendar, the worker can schedule a meeting without disrupting the flow of their current conversations.

Companies like Amazon and Google are already exploring these technologies in the consumer space, with the Amazon Echo and Google Home products; these are screenless devices that connect to Wi-Fi and then carry out services.  This seamless experience puts services in reach of the many people who wouldn’t bother to visit an App Store, or would have difficulty in using a screen and keyboard, such as the visually impaired.

We’ll be looking at some examples of how chatbots and virtual assistants are being used to streamline business processes and interface with customers at the KIN Winter Workshop on the 7^th December 2016.

Machine Learning 

It is worth clarifying here what we normally mean by learning in AI: a machine learns when it changes its behaviour based on experience. It sounds almost human-like, but in reality the process is quite mechanical. Machine learning began to gain traction when the concept of data mining took off in the 1990’s. Data mining uses algorithms to look for patterns in a given set of information. Machine learning does the same thing, but then goes one step further – it changes its program's behaviour based on what it learns.

One application of machine learning that has become very popular is image recognition. These applications first must be trained – in other words, humans have to look at a bunch of pictures and tell the system what is in the picture. After thousands and thousands of repetitions, the software learns which patterns of pixels are generally associated with dogs, cats, flowers, trees, etc., and it can make a pretty good guess about the content of images.

This approach has delivered language translation, handwriting recognition, face recognition and more. Contrary to the assumptions of early research into AI, we don’t need to precisely describe a feature of intelligence for a machine to simulate it.

Thanks to machine learning and the availability of vast data sets, AI has finally been able to produce usable vision, speech, translation and question-answering systems. Integrated into larger systems, those can power products and services ranging from Siri and Amazon to the Google car.

The interesting – or worrying, dependent on your perspective – aspect of machine learning, is that we don’t know precisely how the machine arrives at any particular solution. Can we trust the algorithms that the machine has developed for itself? There is so much that can affect accuracy, e.g. data quality, interpretation and biased data. This is just one facet of a broader discussion we will be exploring at the KIN Winter Workshop, and specifically those deployments of machine learning for decision making and decision support.

Jobs and Skills

The one issue that gets most people agitated about AI, is the impact on jobs and skills. A recent survey by Deloitte suggested 35% of UK jobs would be affected by automation over the next two decades. However, many counter this by saying the idea is to free up people’s time to take on more customer-focused, complex roles that cannot be done by machines.

I think this video from McKinsey puts the arguments into perspective by differentiating between activities and jobs. Machines have a proven track record of being able to automate repetitive, rule driven or routine tasks. That’s not the same as replacing jobs, where routine processes are only part of a wider job function.  According to McKinsey, taking a cross section of all jobs, 45% of activities can be automated, and we’re not just talking about predominantly manual labour. They go on to say that up to a third of a CEO’s time could be automated.

Other research by the Pew Research Centre has said 53% of experts think that AI will actually create more jobs.

The question we need to be asking ourselves is what knowledge and skills do we need to develop now in order to make the most of this technology revolution happening around us and ensure we remain relevant. If organisations don’t find out more about these technologies and how they can be used to improve efficiency or productivity, they can be sure their competitors are!

If you haven’t yet registered for the KIN Winter Workshop – “KnowledgeOrganisation in the ‘Machine Intelligence’ era", (KIN Member link) do so soon! If you’re not currently being affected by AI, you soon will be. Make sure you’re ready!

Steve Dale

KIN Facilitator

Big Data, Data Analytics and AI

Image source: livemint.com

Big Data, Data Analytics and AI have been topics and trends that I’ve been keeping a "layman's" eye on for several years, mainly because I don't like surprises. If I'm going to be replaced at some point by a machine, I'd like to see it coming from a distance rather it sneaking up behind me!

One of the issues I have with with Big Data is just that – the term “Big Data”. It’s fairly abstract and defies a precise definition. I’m guessing the name began as a marketing invention, and we’ve been stuck with it ever since. I’m a registered user of IBM’s Watson Analytical Engine, and their free plan has a dataset limit of 500MByte. So is that ‘Big Data’? In reality it’s all relative. To a small accountancy firm of 20 staff, their payroll spreadsheet is probably big data, whereas the CERN research laboratory in Switzerland probably works in units of terabytes.

Eric Schmidt (Google) was famously quoted in 2010 as saying “There were 5 exabytes of information created between the dawn of civilisation through 2003, but that much information is now created in 2 days”. We probably don’t need to understand what an ‘exabyte’ is, but we can get a sense that it’s very big, and what’s more, we begin to get a sense of the velocity of information, since according to Schmidt it’s doubling every 2 days, and probably less than that since we’ve moved on by 6 years since his original statement.

It probably won’t come as a surprise to anyone that most organisations still don’t know what data they actually have, and what they’re creating and storing on a daily basis. Some are beginning to realise that these massive archives of data might hold some useful information that can be potentially deliver some business value. But it takes time to access, analyse, interpret and apply actions resulting from this analysis, and in the mean-time, the world has moved on.

According to the “Global Databerg Report” by Veritas Technologies, 55% of all information is considered to be ‘Dark’, or in other words, value unknown. The report goes on to say that where information has been analysed, 33% is considered to be “ROT” – redundant, obsolete or trivial. Hence the ‘credibility’ gap between the rate at which information is being created, and our abilities to process and extract value from this information before it becomes “ROT”.

But the good news is that more organisations are recognising that there is some potential value in the data and information that they create and store, with growing investment in people and systems that can make use of this information.

The PwC Global Data & Analytics Survey 2016 emphasises the need for companies to establish a data-driven innovation culture – but there is still some way to go. Those using data and analytics are focused on the past, looking back  with descriptive (27%) or diagnostic (28%) methods. The more sophisticated organisations (a minority at present)  use a forward-looking predictive and prescriptive approach to data.

What is becoming increasingly apparent is that C-suite executives who have traditionally relied on instinct and experience to make decisions, now have the opportunity to use decision support systems driven by massive amounts of data.  Sophisticated machine learning can complement experience and intuition. Today’s business environment is not just about automating business processes – it’s about automating thought processes. Decisions need to be made faster in order to keep pace with a rapidly changing business environment. So decision making based on a mix of mind and machine is now coming in to play.

One of the most interesting bi-products of this Big Data era is ‘Machine Learning‘ – mentioned above. Machine learning’s ability to scale across the broad spectrum of contract management, customer service, finance, legal, sales, pricing and production is attributable to its ability to continually learn and improve. Machine learning algorithms are iterative in nature, constantly learning and seeking to optimise outcomes.  Every time a miscalculation is made, machine learning algorithms correct the error and begin another iteration of the data analysis. These calculations happen in milliseconds which makes machine learning exceptionally efficient at optimising decisions and predicting outcomes.

So, where is all of this headed over the next few years? I can’t recall the provenance of the quote “never make predictions, especially about the future”, so treat these predictions with caution:

1. Power to business users: Driven by a shortage of big data talent and the ongoing gap between needing business information and unlocking it from the analysts and data scientists, there will be more tools and features that expose information directly to the people who use it. (Source: Information Week 2016)
2. Machine generated content: Content that is based on data and analytical information will be turned into natural language writing by technologies that can proactively assemble and deliver information through automated composition engines. Content currently written by people, such as shareholder reports, legal documents, market reports, press releases and white papers are prime candidates for these tools.(Source: Gartner 2016)
3. Embedding intelligence: On a mass scale, Gartner identifies “autonomous agents and things” as one of the up-and-coming trends, which is already marking the arrival of robots, autonomous vehicles, virtual personal assistants, and smart advisers. (Source: Gartner 2016)
4. Shortage of talent: Business consultancy A.T. Kearney reported that 72% of market-leading global companies reported that they had a hard time hiring data science talent.(Source: A.T Kearney 2016)
5. Machine learning: Gartner said that an advanced form of machine learning called deep neural nets will create systems that can autonomously learn to perceive the world on their own. (Source: Ovum 2016)
6. Data as a service: IBM’s acquisition of the Weather Company — with all its data, data streams, and predictive analytics — highlighted something that’s coming. (Source: Forrester 2016)
7. Real-time insights: The window for turning data into action is narrowing. The next 12 months will be about distributed, open source streaming alternatives built on open source projects like Kafka and Spark. (Source: Forrester 2016)
8. Roboboss: Some performance measurements can be consumed more swiftly by smart machine managers aka “robo-bosses,” who will perform supervisory duties and make decisions about staffing or management incentives. (Source: Gartner 2016)
9. Algorithm markets: Firms will recognize that many algorithms can be acquired rather than developed. “Just add data”. Examples of services available today, includingAlgorithmia, Data Xu, and Kaggle (Source: Forrester 2016)

The one thing I have taken away from the various reports, papers and blogs I’ve read as part of this research is that you can’t think about Big Data in isolation. It has to be coupled with cognitive technologies – AI, machine learning or whatever label you want to give it. Information is being created at an ever-increasing velocity. The window is getting ever narrower for decision making. These demands can only be met by coupling Big Data and Data Analytics with AI.

A summary of all the above is included in these slides. 

Trends in Big Data, Data Analytics and AI

I was asked by Managing Partners Forum (MPF) recently to give a brief overview of the current status and industry trends in Big Data and Data Analytics, topics I've been keeping an eye on for several years. The slides are available on Slideshare. The following is shortened abstract from the presentation.

One of the issues I have with with Big Data is just that - the term "Big Data". It's fairly abstract and defies a precise definition. I'm guessing the name began as a marketing invention, and we've been stuck with it ever since. I'm a registered user of IBM's Watson Analytical Engine, and their free plan has a dataset limit of 500MByte. So is that 'Big Data'? In reality it's all relative. To a small accountancy firm of 20 staff, their payroll spreadsheet is probably big data, whereas the CERN research laboratory in Switzerland probably works in units of terabytes.

Eric Schmidt (Google) was famously quoted in 2010 as saying “There were 5 exabytes of information created between the dawn of civilisation through 2003, but that much information is now created in 2 days”. We probably don't need to understand what an 'exabyte' is, but we can get a sense that it's very big, and what's more, we begin to get a sense of the velocity of information, since according to Schmidt it's doubling every 2 days, and probably less than that since we've moved on by 6 years since his original statement.

It probably won't come as a surprise to anyone that most organisations still don’t know what data they actually have, and what they’re creating and storing on a daily basis. Some are beginning to realise that these massive archives of data might hold some useful information that can be potentially deliver some business value. But it takes time to access, analyse, interpret and apply actions resulting from this analysis, and in the mean-time, the world has moved on.

According to the "Global Databerg Report" by Veritas Technologies, 55% of all information is considered to be 'Dark', or in other words, value unknown. The report goes on to say that where information has been analysed, 33% is considered to be "ROT" - redundant, obsolete or trivial. Hence the 'credibility' gap between the rate at which information is being created, and our abilities to process and extract value from this information before it becomes "ROT".

But the good news is that more organisations are recognising that there is some potential value in the data and information that they create and store, with growing investment in people and systems that can make use of this information.

The PwC Global Data & Analytics Survey 2016 emphasises the need for companies to establish a data-driven innovation culture – but there is still some way to go. Those using data and analytics are focused on the past, looking back  with descriptive (27%) or diagnostic (28%) methods. The more sophisticated organisations (a minority at present)  use a forward-looking predictive and prescriptive approach to data.

What is becoming increasingly apparent is that C-suite executives who have traditionally relied on instinct and experience to make decisions, now have the opportunity to use decision support systems driven by massive amounts of data.  Sophisticated machine learning can complement experience and intuition. Today’s business environment is not just about automating business processes – it’s about automating thought processes. Decisions need to be made faster in order to keep pace with a rapidly changing business environment. So decision making based on a mix of mind and machine is now coming in to play.

One of the most interesting bi-products of this Big Data era is 'Machine Learning' - mentioned above. Machine learning’s ability to scale across the broad spectrum of contract management, customer service, finance, legal, sales, pricing and production is attributable to its ability to continually learn and improve. Machine learning algorithms are iterative in nature, constantly learning and seeking to optimise outcomes.  Every time a miscalculation is made, machine learning algorithms correct the error and begin another iteration of the data analysis. These calculations happen in milliseconds which makes machine learning exceptionally efficient at optimising decisions and predicting outcomes.

So, where is all of this headed over the next few years? I can't recall the provenance of the quote "never make predictions, especially about the future", so treat these predictions with caution:

1. Power to business users: Driven by a shortage of big data talent and the ongoing gap between needing business information and unlocking it from the analysts and data scientists, there will be more tools and features that expose information directly to the people who use it. (Source: Information Week 2016)
2. Machine generated content: Content that is based on data and analytical information will be turned into natural language writing by technologies that can proactively assemble and deliver information through automated composition engines. Content currently written by people, such as shareholder reports, legal documents, market reports, press releases and white papers are prime candidates for these tools. (Source: Gartner 2016)
3. Embedding intelligence: On a mass scale, Gartner identifies "autonomous agents and things" as one of the up-and-coming trends, which is already marking the arrival of robots, autonomous vehicles, virtual personal assistants, and smart advisers. (Source: Gartner 2016)
4. Shortage of talent: Business consultancy A.T. Kearney reported that 72% of market-leading global companies reported that they had a hard time hiring data science talent. (Source: A.T Kearney 2016)
5. Machine learning: Gartner said that an advanced form of machine learning called deep neural nets will create systems that can autonomously learn to perceive the world on their own. (Source: Ovum 2016)
6. Data as a service: IBM's acquisition of the Weather Company -- with all its data, data streams, and predictive analytics -- highlighted something that's coming. (Source: Forrester 2016)
7. Real-time insights: The window for turning data into action is narrowing. The next 12 months will be about distributed, open source streaming alternatives built on open source projects like Kafka and Spark. (Source: Forrester 2016)
8. Roboboss: Some performance measurements can be consumed more swiftly by smart machine managers aka “robo-bosses,” who will perform supervisory duties and make decisions about staffing or management incentives. (Source: Gartner 2016)
9. Algorithm markets: Firms will recognize that many algorithms can be acquired rather than developed. “Just add data”. Examples of services available today, including Algorithmia, Data Xu, and Kaggle (Source: Forrester 2016)

The one thing I have taken away from the various reports, papers and blogs I've read as party of this research is that you can't think about Big Data in isolation. It has to be coupled with cognitive technologies - AI, machine learning or whatever label you want to give it. Information is being created at an ever-increasing velocity. The window is getting ever narrower for decision making. These demands can only be met by coupling Big Data and Data Analytics with AI.

Steve Dale (for KIN Enterprise Technology SiG)