BIG DATA WITH DINO PEDRESCHI: THE ROAD AHEAD


The road ahead

Dino Pedreschi, Professor of Computer Science at the University of Pisa and co-lead of KDD Lab - Knowledge Discovery and Data Mining Laboratory, talks about data commons, network effects, Gdpr and our right to explanation.

Prof. Dr.-Ing. Helga Breitner

Data promises to be for the 21st century what steam power was for the 18th, electricity for the 19th and hydrocarbons for the 20th; we are truly experiencing a new industrial revolution. We are also experiencing profound changes in the information technology landscape. The data around us has become so vast, so demanding, there is so much and it is so complex, that it is not about the processing speed anymore; we truly need deeper forms of computational intelligence to make sense out of this new universe of information.

Ruben Gansen

In addition to machine learning and data mining, some of the skills that are required of data scientists include statistics, software engineering, linear algebra, programming languages such as Python and Java, and platforms such as Hadoop for advanced analytics.

YogaFan

In response to Joaquin Oliveras

It’s been a huge decade for big data and artificial intelligence (AI), two of the biggest tech trends we’ve seen this century. From data-driven manufacturing to self-driving cars, we’ve witnessed dozens of jaw-dropping, previously unimaginable feats, all thanks to advances in big data analytics and AI.

Not so long ago, businesses across industries often sat on tons of useful, game-changing data, unsure about the many ways they could put it to use to gain competitive advantage. But as methods in machine learning, deep learning, and natural language processing became more advanced while computing power went up, seemingly useless data suddenly began to make sense.

Joaquin Oliveras

It’s been a huge decade for big data and artificial intelligence (AI), two of the biggest tech trends we’ve seen this century. From data-driven manufacturing to self-driving cars, we’ve witnessed dozens of jaw-dropping, previously unimaginable feats, all thanks to advances in big data analytics and AI.

Анета Владимирова

Because one of the goals of AI developers is to create an artificial system that can fool someone into thinking it is human, the more convincing, personality-consistent generative models have received more attention. These models learn by performing their tasks instead of using a more heuristic approach to match questions to responses (as retrieval based models do), which means they can more easily be misled. Such was the case of Microsoft's Tay. The chatbot turned racist and genocidal after interactions with the public on Twitter helped it learn these traits. Following this, Microsoft took down the bot to make alterations.

Susan Boil

Guys a fun fact, I was on a lecture just the other day, there I had the pleasure to listen some info about a company called chaosgroup. They create software for 3D rendeing, in plain words, their programs create serious animations. And in the theme of Big Data I was told that in the latest avengers movie the character called "Thanos" weights thousands and thousands of terabytes of data.

Greetje Preeti Baart

Big data analytics applications enable big data analysts, data scientists, predictive modelers, statisticians and other analytics professionals to analyze growing volumes of structured transaction data, plus other forms of data that are often left untapped by conventional business intelligence and analytics programs. That encompasses a mix of semi-structured and unstructured data -- for example, internet clickstream data, web server logs, social media content, text from customer emails and survey responses, mobile phone records, and machine data captured by sensors connected to the internet of things. 

Shila Vasuda Gupta

Big Data technologies are evolving with the exponential rise in data availability. It is time for enterprises to embrace this trend for the better understanding of the customers, better conversions, better decision making, and so much more.  It is important for enterprises to work around these challenges and gain advantages over their competition with more reliable insights.

Waclaw Piatek

There is a definite shortage of skilled Big Data professionals available at this time. This has been mentioned by many enterprises seeking to better utilize Big Data and build more effective Data Analysis systems. There is a lack experienced people and certified Data Scientists or Data Analysts available at present, which makes the “number crunching” difficult, and insight building slow.

Sanjeev Jehoram Moriarty

In response to Alex Tetradze

Careen,

What is RDBMS?  Does it have something to do with databases?

A relational database management system (RDBMS) is a collection of programs and capabilities that enable IT teams and others to create, update, administer and otherwise interact with a relational database. Most commercial RDBMSes use Structured Query Language (SQL) to access the database, although SQL was invented after the initial development of the relational model and is not necessary for its use.

Alex Tetradze

In response to Careen Levi

Big data gets generated in multi terabyte quantities. It changes fast and comes in varieties of forms that are difficult to manage and process using RDBMS or other traditional technologies. Big Data solutions provide the tools, methodologies, and technologies that are used to capture, store, search & analyze the data in seconds to find relationships and insights for innovation and competitive gain that were previously unavailable.

Careen,

What is RDBMS?  Does it have something to do with databases?

Careen Levi

Big data gets generated in multi terabyte quantities. It changes fast and comes in varieties of forms that are difficult to manage and process using RDBMS or other traditional technologies. Big Data solutions provide the tools, methodologies, and technologies that are used to capture, store, search & analyze the data in seconds to find relationships and insights for innovation and competitive gain that were previously unavailable.

Neelam Szczepański

Data science, data analytics, and machine learning are some of the most in-demand domains in the industry right now. A combination of the right skill sets and real-world experience can help you can secure a strong career in these trending domains.

Hrœrekr Franzese

Here are the essential data science skills:

  • Strong knowledge of Python, SAS, R, Scala
  • Hands-on experience in SQL database coding
  • Ability to work with unstructured data from various sources like video and social media
  • Understand multiple analytical functions
  • Knowledge of machine learning
     
Sandra Anselmo Escamilla

I want to add a comment on how AI is used in education.  AI can automate grading, giving educators more time. AI can assess students and adapt to their needs, helping them work at their own pace. AI tutors can provide additional support to students, ensuring they stay on track. AI could change where and how students learn, perhaps even replacing some teachers.

Valerija Vroomen

Big data can be contrasted with small data, another evolving term that's often used to describe data whose volume and format can be easily used for self-service analytics. A commonly quoted axiom is that "big data is for machines; small data is for people."

Lucas Jessen

In response to Magdalena Novak

Science and research is currently being transformed by the new possibilities big data brings. Take, for example, CERN, the Swiss nuclear physics lab with its Large Hadron Collider, the world’s largest and most powerful particle accelerator. Experiments to unlock the secrets of our universe – how it started and works - generate huge amounts of data. The CERN data center has 65,000 processors to analyze its 30 petabytes of data. However, it uses the computing powers of thousands of computers distributed across 150 data centers worldwide to analyze the data.

This is fascinating! I have always wondered what the purpose of CERN was, I will read more about it.

Gertruda Filipowski

Industry influencers, academicians, and other prominent stakeholders certainly agree that big data has become a big game changer in most, if not all, types of modern industries over the last few years. As big data continues to permeate our day-to-day lives, there has been a significant shift of focus from the hype surrounding it to finding real value in its use.

Sung-soo Han

In response to Mélina Aubert Porcher

The topic of this discussion is the Road Ahead for big data.  What are some other fields in which big data is slated to make a difference?

Sports.  Most elite sports have now embraced big data analytics. We have the IBM SlamTracker tool for tennis tournaments; we use video analytics that track the performance of every player in a football or baseball game, and sensor technology in sports equipment such as basket balls or golf clubs allows us to get feedback (via smart phones and cloud servers) on our game and how to improve it.

Magdalena Novak

Science and research is currently being transformed by the new possibilities big data brings. Take, for example, CERN, the Swiss nuclear physics lab with its Large Hadron Collider, the world’s largest and most powerful particle accelerator. Experiments to unlock the secrets of our universe – how it started and works - generate huge amounts of data. The CERN data center has 65,000 processors to analyze its 30 petabytes of data. However, it uses the computing powers of thousands of computers distributed across 150 data centers worldwide to analyze the data.

Gaetano Albertini

In response to Ramesh Sigrún Rocca

Melina,

Big data is applied heavily in improving security and enabling law enforcement. I am sure you are aware of the revelations that the National Security Agency (NSA) in the U.S. uses big data analytics to foil terrorist plots (and maybe spy on us). Others use big data techniques to detect and prevent cyber attacks. Police forces use big data tools to catch criminals and even predict criminal activity and credit card companies use big data use it to detect fraudulent transactions.

Got one more: optimizing machine performance.  Big data analytics help machines and devices become smarter and more autonomous. For example, big data tools are used to operate Google’s self-driving car. The Toyota Prius is fitted with cameras, GPS as well as powerful computers and sensors to safely drive on the road without the intervention of human beings. Big data tools are also used to optimize energy grids using data from smart meters. We can even use big data tools to optimize the performance of computers and data warehouses.

Ramesh Sigrún Rocca

In response to Mélina Aubert Porcher

The topic of this discussion is the Road Ahead for big data.  What are some other fields in which big data is slated to make a difference?

Melina,

Big data is applied heavily in improving security and enabling law enforcement. I am sure you are aware of the revelations that the National Security Agency (NSA) in the U.S. uses big data analytics to foil terrorist plots (and maybe spy on us). Others use big data techniques to detect and prevent cyber attacks. Police forces use big data tools to catch criminals and even predict criminal activity and credit card companies use big data use it to detect fraudulent transactions.

Jessica

In response to Mélina Aubert Porcher

The topic of this discussion is the Road Ahead for big data.  What are some other fields in which big data is slated to make a difference?

I thought of one more--city optimization.  Big data is used to improve many aspects of our cities and countries. For example, it allows cities to optimize traffic flows based on real time traffic information as well as social media and weather data. A number of cities are currently piloting big data analytics with the aim of turning themselves into Smart Cities, where the transport infrastructure and utility processes are all joined up. Where a bus would wait for a delayed train and where traffic signals predict traffic volumes and operate to minimize jams.

Jessica

In response to Mélina Aubert Porcher

The topic of this discussion is the Road Ahead for big data.  What are some other fields in which big data is slated to make a difference?

I think one such area is Financial Trading.  High-Frequency Trading (HFT) is an area where big data finds a lot of use today. Here, big data algorithms are used to make trading decisions. Today, the majority of equity trading now takes place via data algorithms that increasingly take into account signals from social media networks and news websites to make, buy and sell decisions in split seconds.

Mélina Aubert Porcher

The topic of this discussion is the Road Ahead for big data.  What are some other fields in which big data is slated to make a difference?

Angelo Faustino Carboni

While AI is critical for self-driving cars, the military, commerce, AI-driven SEO and gaming, it’s poised to make the most human impact in medicine and human behavior. Imagine the UN leveraging neural networks and deep learning to discover what helps some communities thrive and others fall behind. Those lessons can then be leveraged into community builders, city planners, grants and projects.

Darius Tavas

In response to Gaetano Albertini

Big data is used to improve many aspects of our cities and countries. For example, it allows cities to optimise traffic flows based on real time traffic information as well as social media and weather data. A number of cities are currently piloting big data analytics with the aim of turning themselves into Smart Cities, where the transport infrastructure and utility processes are all joined up. Where a bus would wait for a delayed train and where traffic signals predict traffic volumes and operate to minimise jams.

I'd never thought about this but you are right, this is really happening and it's improving our lives. But what happens when technology simplifies our live to a degree when we don't need to go outside, meet with friends and experience nature. Maybe one day we won't even need to walk to get to where we want, not even a simple move and we are there. How is advanced technology going to solve such problems?

Edward Wachter

Technology such as predictive analytics will never be enough on its own to gain meaningful insights from data.  It is essential to apply specialist industry (and sometimes legal) knowledge to the data analytics to identify the factors which have the most impact on potential outcomes.

Prof. Dr.-Ing. Helga Breitner

In response to Анета Владимирова

@Klas, here is an interesting discussion on some of the challenges of Big Data by Dr. Borne.  They all start with the word V: Volume, Variety, Velocity, Veracity, Validity, Value, Variability, Venue, Vocabulary, and Vagueness :)

Aneta, this was a good post about the challenges of big data.  The definition of big data also involves letters that start with V: You have big data if your data stores have the following characteristics:

  • Volume: Big data is any set of data that is so large that the organization that owns it faces challenges related to storing or processing it. In reality, trends like ecommerce, mobility, social media and the Internet of Things (IoT) are generating so much information, that nearly every organization probably meets this criterion.
  • Velocity: If your organizations is generating new data at a rapid pace and needs to respond in real time, you have the velocity associated with big data. Most organizations that are involved in ecommerce, social media or IoT satisfy this criterion for big data.
  • Variety: If your data resides in many different formats, it has the variety associated with big data. For example, big data stores typically include email messages, word processing documents, images, video and presentations, as well as data that resides in structured relational database management systems (RDBMSes).
Gaetano Albertini

Big data is used to improve many aspects of our cities and countries. For example, it allows cities to optimise traffic flows based on real time traffic information as well as social media and weather data. A number of cities are currently piloting big data analytics with the aim of turning themselves into Smart Cities, where the transport infrastructure and utility processes are all joined up. Where a bus would wait for a delayed train and where traffic signals predict traffic volumes and operate to minimise jams.

Fabricio Ruiz

In response to Professor Dodds

I have heard that Blockchains offer an alternative type of data ownership and data marketplace.  Does anyone know what Blockchains are and how they can be used to build a global data commons?  Thanks.

Hello,

The Convergence Ecosystem will lead to a global data commons. It’s not inevitable, but blockchains and distributed ledgers are disruptive technologies that change the structure of the data value chain. Yes, I know that word is overused, but in this case it is true. The point of value capture in the value chain will change. Instead of web companies capturing value and profit by controlling data, data could be stored on decentralised data file systems and blockchains making it accessible to all, not just a select few platforms that collected it.

Dorothea Petrescu

Aneta,


Excellent post.  I just want to add one clarification.  The V-based characterizations you have listed represent ten different challenges associated with the main tasks involving big data.  They are: capture, cleaning, curation, integration, storage, processing, indexing, search, sharing, transfer, mining, analysis, and visualization.

Professor Dodds

I have heard that Blockchains offer an alternative type of data ownership and data marketplace.  Does anyone know what Blockchains are and how they can be used to build a global data commons?  Thanks.

Анета Владимирова

@Klas, here is an interesting discussion on some of the challenges of Big Data by Dr. Borne.  They all start with the word V: Volume, Variety, Velocity, Veracity, Validity, Value, Variability, Venue, Vocabulary, and Vagueness :)

Klas Eriksen

Diversification of opinion is an important issue to Big Data.  We should understand the advantages and challenges involved if we want to preserve and foster democracy in our society.

Please login or register to leave a response.