BIG DATA IS SUDDENLY A BIG DEAL. In late May, at Michael Milken’s annual gathering of financial notables in Los Angeles, one center-stage topic was the impact of increasingly powerful computers and software and their ability to extract meaning from giant data sets. The conference featured one panel, headlined by President Obama’s 2012 campaign manager, Jim Messina, that focused exclusively on big data; Messina discussed how vast amounts of information helped secure victory. But it was a panel on trading that defined the challenge to flesh-and-blood financial and investment analysts posed by the seemingly inexorable march of algorithms, robots and big data.
Louis Salkind, longtime managing director of hedge fund firm D.E. Shaw & Co., framed the challenge of big data most clearly: Salkind, who has a Ph.D. in computer science and robotics from New York University, described a mounting confrontation between analytical machines and securities analysts. Near the end of the hourlong panel, he described a world of increasing automation in which vendors create products “to shred apart Twitter and Facebook” and aggregate trading signals. “Imagine what happens when they start using big-data techniques to look at fundamental data,” he said, recounting a story about a broker who used satellite imagery of Wal-Mart Stores parking lots to forecast quarterly earnings. “When people start integrating these forms of data, it’s just going to be a different world out there.”
Salkind and D.E. Shaw have long been innovators in the use of computers for trading and investment. But some believe that the struggle between automation and human practitioners of finance, which has swept through exchanges, trading floors and even regulation and compliance, has reached a new front: securities analysis.
计算机和软件在跟踪复杂的市场模式或监控和分析新闻报道，社交媒体和其他数字来源中的数据点的优势。虽然他们崩溃了，但电脑永远不必使用浴室或要求提升。它们正在增长越来越快捷，更强大，他们可以获得更多的数据。大数据的使徒预测可以通过卓越的结果解释市场的计算机排名和文件分析师。他们甚至相信大数据将允许机器辨别未来 - 选举，股票价格或公司 - 从噪音中脱颖而出。他们说，大数据不仅可以重塑交易，但长期投资实践。
Others are skeptical. The future is always unpredictable, no matter how much data is aggregated and analyzed, as long as people participate in markets and economies. So far, the practical performance results have been thin: We suffer flash crashes, unhappy financial shocks and bubbles hidden in plain sight. Human analysts cannot process the flood of data that machines can, but they possess something algorithms lack: finely grained, if fallible, judgment. Human analysts can weigh murky values that may not be reducible to quantification; balance long-term and short-term perspectives; profit from intuitions about companies and their futures; forecast the evolution of technologies, brands or fads; and cope with ambiguities and complexities.
这种男人和机器的冲突只是最近的一章，在机械织机取代了基于家庭的旋转轮时加热的几个世纪历史悠久的斗争。虽然结果尚不清楚，但显而易见的是，证券分析世界将在这些强大的新工具的影响下改变。该技术可能很好地改变了证券分析的已经不稳定的经济学，进一步剔除了分析师的行列，将它们划分为能够有效利用新技术和不能的人。大数据可能在这里留下来。更大的问题是，我们如何与之居住？
在这一趋势的核心是算法，一系列步骤或指令，告诉计算机如何搜索和解释数据。这是一个简单而强大的概念，当时与计算机联系在一起。消费者每天都会遇到算法。除了日常任务外，从拼写检查到GPS路由指导到在线购物，算法帮助飞行乘客喷气机并进行医疗诊断，甚至是手术。很快他们可能会开车。
算法也普遍存在财务中，从高频交易到复杂的估值计算到经济预测中的一切中发挥着作用。他们剥夺了在卑鄙的全球财务日常水中洗涤的信息 - 现在在Petabytes或数十亿兆字节中测量的数据。在几周内，没有人类的人类可以在几周内突出这些算法，在第二个部分中，他们需要通过数据流失。
算法Excel在执行快速，几乎无限的计算时。但它们需要数据作为原料。除了传统的财务分析之外，这数据越来越多地提供大量的大量世界：价格，估值，比率。越来越强大的计算机系统挤出新闻项目，财务报表，博客和其他数字文本的市场模式，其中洞察力可能会潜伏 - 一个被称为新闻分析的发展领域。大规模的数字记忆将在数千家公司实时保持标签，以及竞争对手，客户，供应商和投资者。
News analytics weigh factors from arrays of financial ratios to CEO transitions to local work stoppages in rural China for individual stocks, sectors, peer groups or the market. Programs scan horizons and flag events with market implications. Some algorithms even pump out news updates for media consumption.
Fresh data can yield trends and new peer groups tied together by market sentiment, supply-chain relationships or news events. Better yet for investors, novel peer groups often exhibit unique trading patterns.
“Will this source of information change the investment industry? Undoubtedly, the potential is there,” says Wesley Chan, a finance Ph.D.; Goldman, Sachs & Co. veteran; and director of stock selection research at Boston’s Acadian Asset Management, a quantitatively based investment manager that uses news analytics. “Accounting and market data became very important to the investment industry over decades. There’s no reason to think news analytics won’t have the same impact in an even shorter time frame.”
What does this mean for securities analysts? Some advocates of big data believe computing power and predictive algorithms will sweep away traditional analysts and, by extension, a traditional approach to investing. The outcome depends on how effective some of these algorithm-based predictive systems prove to be and whether traditional investment research and active portfolio management can make an effective case for survival.
However, advocates of the continuing centrality of human judgment argue that the binary decisions at the heart of algorithms won’t necessarily make the world more predictable and that the merits of long-term investing will persist. After all, every algorithm has a set of instructions devised by mere people; like economic models, algorithms simplify the world down to manageable inputs and outputs. Human experts command a wider variety of knowledge than algorithms and apply experience in ways that algorithms cannot, says James Owen Weatherall, an assistant professor of logic and the philosophy of science at the University of California, Irvine, and the author ofThe Physics of Wall Street: A Brief History of Predicting the Unpredictable.
医学研究提供了一个经典的例子。1972年的研究要求在生物清单，193年霍奇金淋巴瘤患者的生存期内预测诱发人。专家预测与实际存活时间之间的相关性为零。但是，这些医生产生的活检和通过多元回归模型的编码准确地预测了生存时间。威慑此操作说明了人类和自动角色之间的关键差异。研究人员更好地识别计算机需要运行的变量。计算机可以更有效地综合以系统的方式来制作预测的信息。
在金融中，自20世纪80年代以来，算法在20世纪80年代以来发挥着越来越大的作用。但是作为D.E.Shaw的Salkind建议，有专有算法的公司已经使他们的野心成为购买和持有投资和基本面的雄心。彭博等供应商，道琼斯＆Co.和Thomson Reuters已经加入了一个不断扩大的刚才集合，其中名称Alexandria Investment Research and Technology,AlphaGenius Technologies,Dataminr,数字抹子,卢塞纳研究胜选的rch,MarketPsych,叙事科学,Quantopian,RavenPack和记录未来.
这些公司在新闻分析上销售各种曲折。三大企业汇总互动产品套件。由Thomson Reuters销售的Eikon，部署了在市场趋势，证券价格窗口的用户友好窗口中，即使是海洋飞行器在数字地图上的确切位置。在孤立中，货轮的坐标可能不会改变公司的前景，但与新闻，天气条件和市场需求结合，严重风暴可能会影响市场价值。
同时，启动alphanius矿山推特式市场信号的流量。记录的未来扫除主要新闻出版物，贸易出版物，政府网站和财务数据库，以获得未来事件的明确和隐含的迹象。Ravenpack提供数据产品和先进的可视化工具，识别情绪或媒体关注的国家或公司的高低。寻找市场模式，Alexandria在人类基因组研究中采用计算机技术。Lucena Research，由前F-15飞行员建立，拥有佐治亚州理工学院博士学位。在机器人学中，将人工智能与投资策略相结合。Quandopian用科学家配备了一种发展和反向他们自己的金融算法的手段。
TODAY'S ANALYTIC TECHNIQUES go far beyond the simple word counts that hobbled earlier forays into news analysis and attempts to track market sentiment. Those efforts labeled words positive or negative, anchored in the simplistic assumption that negative words always mean bad news and positive words always mean good. Context, long thought a strength of human judgment, has become paramount.
用于利用新闻分析和市场情绪的窗口不仅仅是在黑暗池中就像股票贸易一样闪光;他们可以，从业者断言，最后几天，几个月或更长时间，从而与投资分析相关。“我已经看到了客户关于我们有关投资视野的信息，长达三年，”Ravenpack和前投资组合经理的定量研究总监彼得哈菲兹说。他的客户寻求数量框架，以获得不需要分析师解释的新闻。“当您获得更强大的新闻分析和更多信息时，您可以绕过分析师并在数据本身中找到价值，”他说。
In February theHarvard Business Review发表了一个案例研究借鉴了录制的未来的情绪检测，这是基于Massachusetts的剑桥的分析启动。该研究报告了一项记录的未来策略，根据情绪归类500股，然后购买了前10％，短缺10％。“RF自身的分析表明，如果投资者在过去一年中遵循其预测和投资标准普尔500指数的股票的建议，他们将大幅超越市场，”根据案例研究。
几十年来的学术研究往往是为了证实爆炸新闻的策略。“资本化的股票，小，无利可图，高波动，非股息支付增长公司或财务困境的公司股票，可能对广泛的投资者情绪敏感，”哈佛大学和杰弗里大学的Malcolm Baker纽约大学的Wurgler报道了2007年的纸张，“股票市场的投资者情绪”。其他研究同意，虽然意见对影响的持续时间和程度不同，但情绪信号可以产生市场变化的预先警告。
To test the value and life span of news analytics, a series of recent research papers by Deutsche Bank focuses on state-of-the-art sentiment analysis. Quantitative strategist Rochester Cahan and his colleagues explore using news — known in the jargon as unstructured data — in stock selection. Their conclusion: The real value in news and Internet data lies beyond simple long positive–short negative sentiment strategies.
Sentiment in absolute terms has less meaning than sentiment relative to market expectations, the Deutsche team concludes. Successful financial models extract alpha from news by capturing complex interactions between sentiment and market data variables like price and volume. “If a company has lots of good sentiment, people writing good things on blogs, tweeting good things, there is an automatic assumption that that is a positive story and you should buy,” Cahan says. “Markets don’t work like that. What matters is expectation.”
情绪为期望提供细微差别的代理。Vale，Bias和Contult的价值，偏见和背景，Tabb集团的高级分析师E. Paul Rowady Jr表示，其中监视新闻分析。根据其背景，裁员可能是良好的或不好的情绪。有些是购买信号;其他，卖信号。福特汽车公司供应商可能会为通用汽车有限公司提供良好的新闻。市场领导者的强劲收入可能会强迫争夺争夺以维持市场份额或相反地冲浪潮流。正确编程，新闻分析算法可以识别上下文中的影响和情绪。
Eventually, Rowady foresees universally accessible data and computational firepower strong enough to absorb everything that is happening in real time and then express market sentiment. “That’s essentially where we are headed,” he says.
CAN新闻分析产品预测市场未来，比传统分析师更好吗？这是一个抵抗明确答案的问题。一方面，新闻分析公司签署协议不会泄露客户的名称，可理解地不想暴露投资策略。（据报道，许多更大的公司一直在使用这种算法工具。）此外，这些分析系统是多种多样的，更快地变化。除了供应商和罕见的推荐方面的反向，还有成功或失败的证据在很大程度上是间接和粗略的。
蓬勃发展，新闻分析必须提升量化的返回高于安装和操作复杂系统的成本。租赁新闻分析系统加上数据源每月可以花费5,000至20,000美元，具体取决于数据刷新的功能，频率和座位数。这对较新的较小公司来说尤其是挑战。
然而，有粉丝。Kevin Shea相信回报超过了成本。“If I can’t get at least 3 to 5 percentage points of alpha from a factor, I’m not interested in looking at it,” says Shea, a veteran of Cadence Capital Management, Batterymarch Financial Management and Invesco who launched Boston hedge fund Disciplined Alpha this year. As its name suggests, the firm adheres to a systematic investment strategy. Its algorithmic approach, developed by Los Angeles–based Alexandria, is rooted in bioinformatics, an information technology that emerged from genomics. Conceptually, analyzing the genome sounds pretty mechanical; after all, DNA may be long, but it only has a four-letter code. But that code features deletions, mutations and “junk” sequences along its 3 billion base pairs, and its interactions with RNA and the assembly of proteins has proved to be extremely complex. Context and relationships matter, just as with financial information.
亚历山大“列车”算法的方式存在上下文元素，以产生市场洞察力。Alexandria在一项研究中分析了55,000名被视为肯定，中立或负面的一项研究中的55,000份文件，而不是将规则分配给分类和对分类情绪的规则分析。谢亚说，当美联储5,000个新文件时，算法将人类评估达到91％的时间。
仍然，一个观察者的91％的命中率看起来像9％的错过率。在很大的卷上，这是很多未命中的，尽管人类评估不一定更好。此外，程序处理成千上万的文件，否则可能会逃脱通知。
没有广泛的股票筛查被反垄断的肯定，正式风险模型和情绪信号，谢伊说，纪律alpha会发现难以在一致的基础上产生令人满意的风险控制性能。至少在纪律处于纪律的alpha中，该算法是仍然展开的实验。
Daniel Sandberg还探索了算法驱动投资的可能性。Sandberg获得了博士学位。在2012年从康涅狄格大学的计算物理化学中，那么，就像他的几个同龄人一样，前往金融。他加入了弗吉尼亚夏洛茨维尔的投资咨询公司的遗产基金会。如果算法可以从科学研究中提取有意义的信号，Sandberg没有理由在金融中不起作用。因此，他开始开发他自己的算法，与冒险支持的波士顿启动Quandopian一起使用，它提供了像跳闸器，数据馈送，算法写作和用户社区等工具。他的第一个项目：一种实现扇区旋转策略的算法工具。
For its part, Quantopian is trying to democratize algorithm development for smaller asset managers like Legacy and even for consumers. It claims to have the first algorithmic trading platform in a browser. And the company is planning to release a discount trading platform, meaning that algorithmic trading, and algorithm development, could be coming to retail investing.
一些怀疑论者继续发现新闻分析少于令人叹为观止。“也许我们在曲线后面，但这不是我们在这里使用的话题，”全球投资公司的股权研究主任说。纽约的投资技术集团为对冲基金提供许多服务和第三方研究;事实上，它的一个营销口号“来自噪音的解码信号”可能来自一个大数据供应商。但ITG没有“新闻分析中的交通”，发言人说。2011年底，伦敦的Derwent Capital Markets在Twitter上推出了一个基于Twitter的基金，在运营的第一个月内庆祝了巨额1.85％的收益。一年后，随着期望的回报，Derwent关闭了基于Twitter的基金。
Human judgment and oversight appear resilient to Christopher Cutler, a former chairman of the Alternative Investments Committee of the New York Society of Security Analysts. “Could this be a big game changer?” he asks. “I wouldn’t overestimate it. Too many things on the fundamental side of investing only humans can take a look at.” By way of illustration, Cutler cites a conversation between an analyst and a corporate executive that yields insight into the mispricing of products by rivals.
T. Rowe Price Group已长期以来，其传统的买入和持有投资策略繁荣。“自动驾驶的过程可能会赚一些钱，但在我的脑海里，这不是魔药，”美国美国股票交易公司负责人安德鲁布鲁克斯说。“有40,000件新闻发布将让您进入LinkedIn $ 6，并保持200美元？我不认为通过阅读大数据来发生这种情况。“投资组合保险在20世纪80年代中期炎热，布鲁克斯召回。然后是1987年的崩溃。是什么派出市场旋转？“投资组合保险，”他说。
新闻分析只有在麦格妮集团的一家香港市场分析师Burke Lau表示，麦格妮集团的市场分析师Burke Lau表示，与Ravenpack合作，销售给全球银行客户的工具。需要人类在算法模型中发现并纠正虚假的置信度（，也许，反之亦然）。一个受欢迎的案例分数：当电子交易骑士骑士首都集团的新软件于2012年8月出现故障，释放洪水在交易所上的购买订单，在人类或人类的人类或人类之前基本上消失了数百万美元 - 不得不进行干预。
计算机非常适合找到特定的数据集和预测结果。但是，当意想不到的出现时，电脑均为没有人为干预的情况而生病。例如，IBM Corp.的国际象棋电脑，深蓝色，击败俄罗斯冠军加里卡斯帕罗夫，但仍然无法在扑克中露出一个虚张声势 - 一个类似于棋盘的游戏。
“在国际象棋中，你可以在需要了解的一切周围画一个圆圈，而谁知道什么会影响汽车行业？”Leslie Valiant说，T.Jefferson Cocidge Im哈佛大学计算机科学和应用数学教授，2010年上午A.M.图灵计算卓越研究奖。电脑看到棋牌游戏中的一个壮观的举动，但截至目前，勇敢说：“我们不知道如何使它们复制常识。”
算法在他们可以在最小的人类干扰提供投资建议之前有更多的学习。但是，分析师希望判断总是胜过算法可能面临着一个粗鲁的惊喜。当智能手机所有者可以作证时，自然语言处理允许算法分析日常英语，取得了进步。读扑克诈唬需要对语义细胞的敏感性。使用多种算法的Deew Blue的IBM继任者命名Watson，着名的人类冠军Jeopardy!partly by accurately parsing the game show’s wordplay.
“The advent of semantic search in financial markets stands to force a shift in the way financial professionals consume and analyze information and, most important, make money,” says Haris Husain, who heads a Thomson Reuters effort to develop intelligent search tools rooted in natural language.
And that may prove to be the rub for securities analysts, who have already suffered through more than a decade of wrenching change. With investment capital fueling start-ups and lots of shrewd engineering minds engaged in devising more-intuitive algorithms, securities analysts may face significant disruption. The best will survive and prosper; the rest may fall to the machine.
Christopher Steiner进一步走了很多。施泰纳，作者自动化这一点：算法如何统治我们的世界，是前者福布斯技术编辑和当前的互联网企业家;他不仅对展望进行了证券分析师，而且是专业的投资组合经理。“在十年中，我没有看到一大批余地的钱，”他宣称。
That may verge on the glibly apocalyptic, but there clearly are larger forces at work throughout the white-collar economy. Nobel laureate and Princeton University economist Paul Krugman echoed the warning in arecentNew York Times柱子旨在思考高级学历的工人意味着工作保障。“他写的技术对劳动力的影响更为黑暗的照片，”他写道。“在这张照片中，受过高等教育的工人可能与受过教育的工人一样可能发现自己流离失所和贬值。”
在这种情况下，投资研究可能需要较少数量的META-或ÜberAnalysts，说明在5月份会议的大会上召开会议的量级分析师Joseph Mezrich表示，这是一个专门的小组。
尽管如此，这些是新闻分析中的早期日，MIT Sloan Management Monance教授Andrew Lo教授，他指导学校的金融工程实验室。LO看到可以从新闻中提取数据的算法看涨前景。他说，把新闻分析工作到工作，或者在市场搬家时投降无数的机会。“无论你试图预测市场行为多少，”罗说，“新闻量将始终是这种情况会产生波动性的尖峰。”
What we need to know is whether that news-driven volatility is just a passing fancy or a substantive development, a summer storm or climate change. If only we had an algorithm to tell us that. • •