As the newest member of the IOWA.rocks partner ecosystem we are delighted to welcome CubeLogic’s CEO, Lee Campbell to share his thoughts around how and why CubeWatch enables users to utilise the mass of unstructured data available on social media channels.
The impact of social media over the last ten years has been dramatic. The number of daily tweets is now around 500 million. Facebook has an even larger amount of daily activity and other sites such as LinkedIn are also generating an enormous amount of content. These channels provide vast quantities of live, continually updating data and most of us are already using these channels to enable better communication with our target audiences or stay abreast with our industries and competitors. But the big question is, could the data be captured and stored in a such way that would provide deeper insights into market activity, trends and sentiment for the trading community?
Consider the impact of….…
- Political instability or unrest in a major oil producing country, or news of a specific cut in oil production in a particular region or within OPEC.
- Negative opinion spreading very quickly across social media and this in turn can affect a company’s credit worthiness.
- Adverse weather conditions which could seriously affect commodity prices, as the need for heat or power consumption is driven by climate across most regions.
All of these indicators are very likely to be available social media, but all too often buried in a sea of information. Just imagine the potential to improve overall business performance if we could continually gather all of this valuable data and use automated processes to; efficiently organise it, filter out irrelevant content, manage multiple languages, different spellings and acronyms as well as stripping out hashtags or anything else which could be misleading. And this is no small task… Big data requires complex algorithms and extensive language libraries in order to make any sense of it.
Credit risk management would benefit greatly from sentiment and insight from an organised social media data set. If the data is streamed in real-time, filtered for relevance and absorbed into an advanced topic model, it can be used to provide timely information for credit risk managers to analyse and monitor specific counterparties. This would then proactively prevent the impact of future credit risk events on the portfolio. Newsfeed type data could also be very useful and deliver greater insights into a particular company where information may be only available from say, a previous year’s financial statement or a public ratings agency.
Market risk managers and traders need up to the minute information to assess the impact of open positions and formulate trading or hedging strategies to reduce the risks inherent in those positions. Social media could provide timely, insightful information a particular market or commodity. If this data can very quickly be streamed, organised and presented to a trader or risk manager, it would enable faster, more secure trading or hedging capabilities. Social media information could also contain valuable insights into market, geographical or climate related events which will in turn could have a significant impact on the price of a financial instrument or commodity. Knowing the origin of a specific tweet or post could provide invaluable insights into specific activity in a particular region or pricing hub.
Machine learning and artificial intelligence (AI) as a technology is becoming the norm as computing power increases and algorithms become more sophisticated. This captured and well organised organised social media data could potentially be used in machine learning algorithms to predict future events, discover trends and provide trading recommendations ahead of changes in price curves. However, it’s important to understand that different models will provide varying degrees of insights based on their complexity and sophistication. The accuracy of a sentiment model will depend on the quality of the subjectivity lexicons which are pre-built in the library. In other words, the number of negative and positive lexicons that have been defined has a major impact on the accuracy of the model. Another consideration is that this type of model is not a context-aware model; tweets that have a sentence containing: “amazingly bad” may be catalogued as neutral as “amazingly” may be considered positive and “bad” negative. More advanced, context-aware models will ultimately provide better results.
Over time when looking at social media information in relation to a particular country or counterparty, a sophisticated machine learning algorithm could be used to detect subtle changes in mood or sentiment that otherwise may be overlooked. A trend of slowly increasing negative or positive sentiment might imply something could happen in the future.
It is clear that the relevance of big data in all areas of our lives has become very significant. The technology available to consume and make sense of this information is now more freely available and better understood. The challenge for consumers is having the ability to organise the information in a meaningful way and then visualise it, and also how to filter out the ‘noise’ to quickly get to the relevant trend or sentiment that is of interest to the daily business. In trading and risk management, the importance of this new data source is only now only just being realised. The emergence of machine learning, coupled with immediate availability of quality, relevant information from social media, and the application of business intelligence visualisation and reporting, has the potential to change the face of risk management in the not too distant future. If technology companies can harness this data to provide risk managers more proactive risk management techniques and insights, this will bring enormous benefits to the business almost immediately. We believe it is an important component of the future business intelligence enabled risk management landscape.
If you would like to contribute an article to IOWA.rocks then please contact our community manager.