In Depth: How important is big data?

If you want proof that big data is a big thing, just look at the jobs pages. Inside, you'll find desperate appeals for statistics-savvy data scientists, and the salaries are often eye-watering.

They're evidence that businesses in the UK and beyond are looking at ways of using the data they already collect – whether that's about customers, their industry or about financial markets – with as much efficiency as possible.

Digital markers

Data is everywhere, it's doubling every two years, and it's fast becoming a business' biggest resource. The digital markers we all create – whether a GPS-tagged photo, check-in on Facebook or a click on a website – could last forever.

So far, we've got about 1.8 zettabytes of data on planet Earth, but it's how we link up, analyse and use all of this data – and much more besides – that's going to completely change our world. And that means business.

Some say that by mastering big data, companies can stop wasting time on useless initiatives and pitches that go nowhere - and increase margins by a whopping 60 per cent.

Even if it's a fraction of that figure, harnessing big data properly can make the difference between a profitable business and a non-starter.

Big data is big money. ABI Research states that global spending on big data by organisations exceeded US$31 billion in 2013, and will reach a staggering US$114 billion in 2018 – and that includes salaries as well as hardware and software.

It's a data-rich, skills-poor environment at present, but the future will be one of increasingly sophisticated hardware and ever-better data analytics that collect, calculate, predict and much more besides.

The iceberg problem

Unfortunately, big data is scattered. It's estimated that IT departments spend about 70 to 80 per cent of their time maintaining existing systems and databases, largely because there are so many of them.

Separate, created for narrow purposes, and often based on databases that don't talk to each other, it's what's known as 'the iceberg problem', where systems can't exchange big data, which therefore goes untapped.

"One of the companies we're working with has 6,670 'icebergs' used by various arms of the business," says Dr Andrew Sutherland, Senior Vice President of Technology at Oracle EMEA.

"Some are very important, each one was built for a reason, and all had and still have a business case; and they were all built by very clever, experienced people – but each application has been built to fulfil one need, and they use completely different databases."

False economy

It's a common problem, but businesses who've invested a lot in such icebergs often don't want to replace them. For a business wanting to embrace big data, that viewpoint represents a false economy.

Petabytes of big data are now flooding in to businesses and a consistent, common platform is required, capable of exchanging data and running apps – whatever their source – using only one stack that can be managed, maintained, recovered and backed up as one.

It's all about consolidation. "Whether you're a police force, a gas extractor or a retailer, there will be areas of your business that you want to be able to do more efficiently or cleverly – and you can only do that if you consolidate and simplify your IT, so you can concentrate on the apps that add value to your business," says Sutherland.

One problem here is that there often exist huge teams of IT consultants whose job it is to specify and commission more icebergs; asking them to give their blessing to using one unified, interoperable system that best takes advantage of big data is like asking turkeys to vote for Christmas.

But if you do get big data into one place, what happens to it next?

Machine learning & super-computers

The big data revolution will be led by machine learning and super-computers. They've been called the 'coal in the furnace' of the internet that will drive Web 3.0, but the sole reason for the new generation of artificially intelligent super-computers is the massive rise in personal and business data.

Eventually enabling websites and apps to track and predict our every action and desire, super-computers feed off big data.

Reasoning, perception, social and even some degree of cultural awareness are the selling points for Watson, IBM's super-computer that runs on 16 terabytes of RAM, and brings a much-needed injection of artificial intelligence to big data analytics.

Its smart learning DeepQA software is able to search databases, spot patterns and make predictions, which could revolutionise the financial and medical industries, both of which are currently drowning in data.

Big data in medicine

As well as reducing the need for investment bankers, big data analytics could also banish bad decisions by doctors.

"Watson could be used to search through millions of pages of academic research and drug trials, something a human could never do or keep up with, and in double-quick time," says Joe Peppard, a professor at the European School of Management and Technology in Berlin, Germany, which consults on IT strategy for large corporations.

"It could provide a doctor or nurse with the required diagnosis and even the most appropriate medication plan, having taken into consideration all the latest thinking on any specific health problem."

Call centres streamlined

Such data analysis could also be used to vastly improve call centres, where about 60 per cent of enquiries end in frustration for callers.

"Watson is able to search a far greater amount of data in a much faster time," says Peppard, "so in many cases it could pre-empt enquiries so that the answer is available before the question has been asked."

"Machine learning, and its application in advanced analytics, is one area that will make both the public and private sectors data-savvier than anything we've seen so far," says Dan Shey, Practice Director at ABI Research.

"Big players such as IBM and HP are understandably moving to this direction, but at the same time we can also see analytics startups, like Ayasdi and Skytree, that have machine learning in their very DNA. Eventually, such innovations will put analytics within any domain expert's reach. At that point, data will stop being big."

Uncovering the Universe

Harnessing big data properly can mean new discoveries and possibilities, and nowhere more so than in deep space.

The European Space Agency's Gaia mission to create a complete three-dimensional map of the Milky Way involves taking super hi-res photos of the heavens, but to make any use of this data requires a powerful information system.

Archiving and processing scientific data collected by the Gaia satellite, the Italian National Institute of Astrophysics (INAF) uses a data management system from Oracle to store its "very precious heritage of astronomical data that will have to be stored for the whole 21st century and beyond," says Roberto Morbidelli, Scientific Operation Manager at INAF.

Processing petabytes of big data from Gaia is a huge task, but Oracle has history; it also works with the operator of the Large Hadron Collider, CERN.

The sources and uses of big data in business are now being revealed, but if we know one thing it's that the hyper-efficient collection and analysis of data is already becoming a major tool.

"In five or 10 years' time, big data will not be a source of competitive advantage, it will just be a necessary part of normal business," says Eddie Short, UK and EMEA Leader for Data and Analytics at KPMG Management Consulting. "Without exploiting it, a business will die."

The Hidden Galaxy S5 Feature Parents Will Love

The Hidden Galaxy S5 Feature Parents Will Love is a post by Travis Pope from Gotta Be Mobile.

New software screenshots include an as-yet-to-be announced feature that could finally give smartphone buyers a decent reason for buying the Galaxy S5 and a Gear smartwatch on release day. Pairing a Galaxy S5 with a Gear smartwatch turns the upcoming high-end smartphone into a baby monitor.

The feature’s details were leaked this past week by SoyaCincau. Named the “Baby Crying Detector,” the featured hides in the Accessibility menu within the Galaxy S5’s settings. Users with a Samsung Gear smartwatch will be able to pair the two and have the Gear alert them when their Galaxy S5 detects sounds. For this to work the parent will need to have left their Galaxy S5 next to their child.

Of course, a feature concerning children and health is only as good as those using it. As such, parents will have to keep a close eye on their Gear smartwatch for updates concerning their child. They’ll also need to make sure that they can feel the Gear’s vibrations too. Both are issues that the Baby Crying Detector mentions before activation.

In fact, Samsung goes as far as to make users explicitly agree that they use the function “at their own risk.” The agreement also says that Samsung isn’t liable for “any loss, damage or injury” in the event that something goes wrong. While it’s certainly a bit troubling, many baby monitors already include that sort of language with their user manual.

The Galaxy S5’s Baby Crying Detector sounds like a differentiating feature that most parents would be happy to have on their next smartphone. It’s a unique value addition, something that parents of small children could appreciate over time, though it’s unclear whether anyone would specifically purchase a Samsung Galaxy S5 for the feature alone.

That users will have to pay $299 for the Gear smartwatch plus the Galaxy S5 itself might be a bit too much for parents to stomach.

The Samsung Galaxy S5, the successor to the very popular Galaxy S4, will go on sale in the United States for $199 with a two-year contract on April 11th. Users who pre-order from Sprint or T-Mobile today won’t have to put any money down.

Microsoft's Satya Nadella reshuffles senior ranks

 Microsoft Corp on Monday named three executives to its cloud, devices and Xbox gaming divisions in one of new Chief Executive Satya Nadella's first management shakeups. 

In a public email, the world's largest software maker promoted Scott Guthrie to head of the cloud and enterprise group, the $20 billion division that Nadella helmed before replacing Steve Ballmer as CEO in February. 

Former Nokia Oyj chief executive Stephen Elop - at one time considered a candidate to replace Ballmer - will join Microsoft as executive vice president of the devices group, reporting to Nadella, after Microsoft closes its $7.2 billion acquisition of the Finnish device manufacturer. The two companies announced the transaction in September. 

Microsoft also named Phil Spencer to lead its Xbox operation, reporting to operating systems group chief Terry Myerson. As the head of hardware, Elop will also play a role in developing the Xbox, Nadella said in the memo. 

Speculation has mounted for years over the fate of the Xbox business, which has been deemed a potential candidate for a spin-off by some analysts and Microsoft insiders. Bloomberg reported last year during the CEO search that Elop, then the front-runner to replace Ballmer, favored a sale himself, although his thinking may have evolved since. 

Nadella, who unveiled Office for Apple mobile devices at his first public appearance last week, signaled that he would shift the company's emphasis away from prioritizing its Windows operating system at the expense of other initiatives. 

"As I said on my first day, we need to do everything possible to thrive in a mobile-first, cloud-first world," Nadella said in an email. "The announcements last week, our news this week, the Nokia acquisition closing soon, and the leaders and teams we are putting in place are all great first steps in making this happen." 

Wipro in talks with colleges to launch analytics courses

As demand for talent in niche technology areas such as data analytics gets fierce, some software companies are turning to universities and colleges to keep up with the digital revolution. Wipro Technologies, India's third-largest software provider, is in talks with several engineering colleges and universities to introduce courses in data analytics as part of their curriculum. 

Its goal: to expand analytics talent pool in the country and adapt to the changing business and technology landscape in India's $108-billion information technology industry. "The reality is that Indian companies are scrambling for good talent in areas such as analytics, cloud and mobility," said an industry source with direct knowledge of Wipro's initiative, which is expected to be in the final stages of approval. 

Once introduced, the programme will help Wipro introduce tailored courses in topics that are important to its ongoing business and reach out to a wider talent pool at the time of campus placements. "It's the first of its kind in the Indian software sector, although students who undertake the programme are under no obligation to work for Wipro," the source added. 

The initiative is an extension of the company's Wipro Academy of Software Excellence (WASE), where it offers science graduates an opportunity to do a Masters degree from the Birla Institute of Technology in Pilani. ET couldn't independent verify the names or number of colleges where Wipro will offer courses. 

The move comes at a time when India's top software outsourcing companies are trying to build expertise in "Smac" - the acronym for social, mobility, analytics and cloud - that is expected to be a major source of their revenue in the coming years. 

Partnering with Startups to Expand Expertise 

As traditional outsourcing services such as application development, maintenance and testing gets commoditised, these companies expect their clients in the US and Europe to step up investments in new technologies, but the dearth of talent worries them. 

A 2011 Mckinsey report predicted that IT industry will face a significant shortage of analytics talent, particularly of people with deep expertise in statistics and machine learning. The consulting firm said it expects a 50 per cent to 60 per cent gap between the supply and demand of people with deep analytical talent. 

 

Over the past few quarters, software services companies such as Tata Consultancy Services, Infosys and HCL Technologies have been partnering with startups and running in-house training programs to expand their expertise in these technologies. While Wipro recently partnered with Chennai-based startup Orangescape, Noida-based HCL Technologies, India's fourth-largest technology firm, works with several startups. 

Technology advisory firm Gartner estimates that the global business intelligence -- which includes analytics -- market will be $17.1 billion (Rs 1,02,000 crore) by 2016. TCS, India's largest software provider, recently organised a two-day gathering with college deans and principals to speak to them about what the industry wants from academic institutions. Infosys, HCL Technologies and Mindtree said they do not have such partnerships at the moment, while a Wipro spokeswoman declined to comment. 

In a note last week, Mumbai-based brokerage firm Kotak said Wipro is working on three large analytics contracts, each with potential to return $50-100 million over 3-5 years. Wipro gets 7.2 per cent of its nearly $7-billion (Rs 42,000) revenues from analytics practice, the brokerage said. 

Data analytics company Mu Sigma in 2012 set up the Mu Sigma University that picks fresh engineering, computer science, and science grads from Tier 1 & Tier 2 colleges across India and the US and puts them through a training program in analytics. "The kind of talent - people with the right skill set, tool set and mindset - that are required to address the challenges businesses face has been hard to find," said a Mu Sigma spokesperson.

IBM in talks with telcos for using data analytic tools

Technology major IBM is in active talks with India's mobile phone carriers for implementing data analytics tools to help service providers get more revenue out of existing customers and improve their product offering by better understanding customer demands.

With telcos sitting on tonnes of information about each mobile phone user, Vikas Sehgal, IBM India and South Asia's director for telecom industry, feels authorised information can be tapped into for giving more intuitive and tailor made solutions to customers.

  "The technologies to bring data from network, call records, web, social media use and internet use into a mode where it can be churned and exploited to take decisions that are much deeper and informed, are picking up," he said.

He added telcos were adopting new technologies but for different reasons. For instance, some want to know which smartphones subscribers use to download what kind of content or which over-the-top (OTT) applications are subscribers accessing on the network.

IBM India and South Asia's chief technologist, Ramesh Gopinath, added telcos sit on large amounts of data that they can't touch, tremendously increasing the risk of them becoming 'dumb pipes', unless they leverage data to understand customers better.

Gopinath, who is also director of IBM Research India, said the company had built Vibes, a tool that can help telcos to identify communities of like-minded people that can be targeted for specific promotions. People in Motion, another tool, can be used to create trajectories of movements of subscribers based on their locations, which can in turn be relied upon to offer promotional offers.

Globally, such services based on 'geo-fencing' are being experimented with for giving customers specialised services, including discount coupons or specific deals from a retailer in a small geographical area, for instance a mall. Gopinath predicts video consumption among Indians to rise manifold with smartphones becoming cheaper and Indians demanding low-cost high speed internet from telcos, which lead to higher costs as telcos upgrade their networks to manage the massive pressure. IBM has created Async that can delay high bandwidth consumption, for example app updates, to a time when the telco's network is least congested.

Snapdeal may surpass Flipkart in sales this year

Snapdeal is literally snapping at Flipkart's heels. Its sales might even be ahead of its older and better known e-commerce peer.

Snapdeal's founder and CEO Kunal Bahl indicated to TOI that it would hit $1 billion in sales (or gross merchandise value, as the ecommerce world calls it) in 2014, a year ahead of target. Flipkart, based on its February numbers, had said earlier this month that it had touched an annualized run-rate of $1 billion, again indicating that the full year 2014 would see the company achieve that landmark.

Bahl said that in 2012 Snapdeal had said it would do $1 billion in 2015 calendar. "Now there is a very optimistic chance that we will beat that very significantly. We will be the fastest and the most capital efficient company to hit $1 billion, and by a pretty significant order of magnitude. And this would be very soon," Bahl said.

Snapdeal was founded in 2010, three years after Flipkart, by Bahl and his partner Rohit Bansal. It started as a daily deals site selling meal and spa vouchers but morphed into an online marketplace in 2011, now offering an assortment of 4 million products from over 20,000 sellers. Recently, the world's largest digital marketplace eBay led a $134 million (Rs 830 crore) investment in Snapdeal.

Bahl said Snapdeal grew 500% in the last 12 months. He attributed the extraordinary growth to two things. One, unlike Flipkart, it avoided the expensive model of building its own inventory, and instead created a platform that allowed a retailer anywhere to offer his products to buyers anywhere (known in e-commerce parlance as the marketplace model). Snapdeal focused also on ensuring efficient delivery. This model, Bahl said, resulted in a much more efficient use of capital too.

Two, it looked beyond India's major metros, even as most e-commerce firms focused on these centres. "Our hypothesis was a little different. We said we could set aside this 1 million audience of urban elite, and focus on the millions of consumers in Middle India who aspire to buy double door refrigerators and washer-dryer combo washing machines. Today, 50% of our orders come from tier 2 and 3 cities," said Bahl.

Snapdeal believes large appliances and fashion are important product categories for Middle India. "India is a heterogeneous market and access to products is limited. Though summer hasn't started yet, we are already selling 250 ACs a day in tier 2 markets. In April, we should be shipping over 25,000 ACs. We are also selling 70-80 fridges and 50-60 washing machines a day," Bahl said.

 

Fashion alone is expected to touch Rs 1,000 crore in sales in the next one year. "Over 60% of all units sold on Snapdeal are fashion goods. About 15 months ago, it was zero. It's just that we sell different things. The fluorescent green Nike shoe that other sites offer, the economics of selling those are worse than selling a mobile phone. About 90% of our assortment is not comparable to any site in India. We don't focus on brands; we focus on the long tail of other offerings," Bahl added.

Snapdeal adds a new product every 20 seconds on its portal and plans to have 8-10 million products in its online catalogue by 2014-end. It also plans to double the number of sellers to 50,000 in the same period.

Research firm Forrester estimates that India's e-commerce market will touch $8.8 billion in 2016, though some say it could be as high as $16 billion by then. The country's internet users have risen to 200 million with over 20 million of them transacting online.

Apple's 'transparent texting' makes texting while walking safer

US tech giant Apple has filed a patent for new technology that aims to make texting while walking safer by replacing the text background with a live video feed of whatever is in front of the smartphone user.

Many smartphone users keep their eyes glued to the screens while texting and walking which can cause them to stumble over or bump into obstacles.

Apple's 'transparent texting' aims to solve this problem by overlaying messages on a live video feed from the rear camera of the smartphone.

This will allow users to see what is happening beyond their phone and text at the same time.

"A user who is walking while participating in a text messaging session may inadvertently collide with or stumble over objects in his path because his attention was focused on his device's display instead of the path that he was traversing," according to the patent filed in the US.

The patent describes how the transparency feature could be activated by pressing a transparency button within a texting session that switches on the video feed.

The feature would then ensure text bubbles appear overlaid over the live background video. The text bubbles themselves could be opaque or semi-transparent, according to Tech Crunch.

The patent, originally filed in September 2012, details potential extensions of the concept of transparent texting, including replacing the background of a webpage with a live video feed, so that the text of a website is overlaid over whatever environments the device user is moving through.

"Alternative embodiments of the invention can be applied to virtually any computer-executable application in which text is presented over a background," the patent application said.

A recent study by researchers from University at Buffalo in the US found that texting while walking may result in more injuries per mile than distracted driving.

The study found that though injuries from car accidents involving texting are often more severe, physical harm resulting from texting and walking occurs more frequently.

Rise in your profession: Tips on how to utilize an alumni network

Why do you need to network with the alumni? Unlike the personal front, where the family is a readymade set of relationships, the professional front does not offer a starting point.

To move up professionally, you need to build a support system of friends and allies among like-minded people. Your network of alma mater friends and the alumni from companies with a strong culture provide a pool of people seeking to connect and help.

 

 However, this is not available on tap and you will need to build it. Here's how.

Don't take alumni for granted

Invest in relationships. Harsh Sapru, a sales manager, applied for an advertised role with a leading Indian FMCG firm. When he found that a regional manager was an alumnus from his MBA school, he sent him an e-mail, requesting to be short-listed to the interview based on their common background.

The manager forwarded the e-mail to the HR, which promptly moved Sapru's resume to the trash folder. Sapru had taken the alum for granted and had called for an unreasonable favour from a stranger. This irritated the alum, who had Sapru removed from the selection. The right way to work with alumni is to build relationships early in life.

Don't cold call them when you are serving your notice period. Start by committing three hours a week to relationship building.

Call up alumni in your immediate circle and speak to them with no other agenda but to connect. Or, fix up a dinner with a group of alumni friends and count that time against the three hours. Thereafter, widen your network and reach out to people through social media like Linkedin and Facebook.

Drop in a line and make that odd call to speak and share common stories or interests. Over a year, you would have invested more than 150 hours and built a strong web of ties.

Be visible

People will know who you are. After graduating from school, Delhi-based Gaurav Verma became the unofficial connecting point for batchmates.

From compiling and sharing contact details and job/education updates to organising well-attended school reunions in his city, he seemed to be always there for everyone. In his late thirties, when Verma decided to become an entrepreneur, he reached out to his network for investors, distributors and customers.

They responded immediately and his venture took off. Being visible is not too challenging. Try attending all alum meets and events, visit your alma mater to greet your teachers and professors, and walk into your old hostel to host an interactive business session for students.

Similarly, get involved in discussions on social network groups and become an organiser. Create Facebook discussion groups over common interests and organise meet-ups where you invite speakers of interest.

First give, then receive

Offer to help people without any transaction involved. For instance, you could lead your firm's recruitment drive to your engineering college, or coach unsuccessful students on clearing aptitude tests, group discussions and interviews, even offer unpaid internship to students to help them gain experience.

 

 Adopt a giving attitude in your alum network and it will build goodwill that will pay you back with interest. Offer to host alumni for the night when they are passing thro ugh your city.

Connect fledgling alum entrepreneurs to potential customers. Donate money to your alma mater for scho larships to needy students. Meet alumni who request advice and share references when asked.

Help out students with applications for higher studies. The opportunities for giving back to your network are endless, and the returns huge.

 

Research on alumni

Know your alumni well. When connecting with them and building a network, it pays to conduct detailed research. Moving to a new location? Find and reach out in the area to the alum who share similar interests, including hobbies, friends, industry, etc.

Similarly, to grow within your industry or profession, use your social networks to find and connect with alums who are working in the same sector. Then share information, help and form connections. Don't forget to maintain an Excel sheet for your research.

Seek advice, not jobs

During your job search phase, ask for references or advice only. If you are not close friends with the concerned alum, you do not have the space to ask for a job vacancy or offer on the strength of a nonexistent relationship.

Walking up to an alum and requesting a job makes him feel cornered and helpless. Recognise that an individual in the corporate world cannot hire without due process.

A better way to approach is to share that you are looking for a job and then ask for advice on the industry/firm/role, or for references you can talk to for advice. Now the alumni can easily share inputs and walk away satisfied from the interaction.

Avoid cold calls where possible and seek to meet face to face through mutual contacts or alumni/industry functions. You may then leverage the advice to find a suitable opportunity.

Want to quit smoking? Listen to your smartphone

Smartphones and tablets may hold the key to get more clinicians screen patients for tobacco use and advise smokers on how to quit, research shows. 

In the study, using mobile phones loaded with tobacco screening guidelines prompted nurses to ask patients about their smoking habits. 

After the counselling, most of smokers expressed a willingness to kick the habit, according to a study from Columbia University School of Nursing. 

"These findings suggest that mobile applications can play a significant role in curbing tobacco use," said lead study author Kenrick Cato, an associate research scientist at Columbia Nursing. 

The study evaluated tobacco screening rates for more than 14,000 visits at clinics in New York City. 

While overall screening and counselling rates were increased by use of the mobile tools, the gains varied by race and gender, the study found. 

The technology can serve to remove any unintended bias clinicians might have about which patients are most likely to benefit from intervention, Cato noted in the study published in the journal Oncology Nursing Forum.