Auto Generating Blog Posts with RapidMiner
I designed a RapidMiner process that downloads the historical stock data using three macros: symbol, start_date, and end_date.
Lately, I've been fooling around with the new Blot.Im, a blogging engine and decided to see if I could use RapidMiner and some Javascript to auto-generate a blog post with a stock chart. Since Blot can parse text, markdown, and HTML files, I decided to see if I can get this to work with an HTML. [While Markdown has the ability to parse HTML tags, it can't parse Javascript correctly][1]. After some pain, I got it to work with Blot, but you can extend this to any type of blogging system that parses HTML.
I designed a RapidMiner process that downloads the historical stock data using three macros: symbol, start_date, and end_date. These macros help set the parameters in the Yahoo Stock Data operator but were also used to pass through to the actual Javascript code.
For the stock chart, I used the plot.ly library and passed the %{symbol} macro and %{end_date} to the script. For the sake of syncing up the saved CSV file, I had to be careful that it was appended with those macros.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="7.0.001">
<context>
<input/>
<output/>
<macros>
<macro>
<key>symbol</key>
<value>VSLR</value>
</macro>
<macro>
<key>end_date</key>
<value>2016-04-14</value>
</macro>
<macro>
<key>start_date</key>
<value>2015-10-14</value>
</macro>
</macros>
</context>
<operator activated="true" class="process" compatibility="7.0.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="quantx1:yahoo_historical_data_extractor" compatibility="1.0.006" expanded="true" height="82" name="Yahoo Historical Stock Data" width="90" x="45" y="34">
<parameter key="I agree to abide by Yahoo's Terms & Conditions on financial data usage" value="true"/>
<parameter key="Quick Stock Ticker Data" value="true"/>
<parameter key="Stock Ticker" value="%{symbol}"/>
<parameter key="select_fields" value="CLOSE|OPEN|DAY_LOW|DAY_HIGH"/>
<parameter key="date_format" value="yyyy-MM-dd"/>
<parameter key="date_start" value="%{start_date}"/>
<parameter key="date_end" value="%{end_date}"/>
</operator>
<operator activated="true" class="sort" compatibility="7.0.001" expanded="true" height="82" name="Sort" width="90" x="179" y="34">
<parameter key="attribute_name" value="Date"/>
</operator>
<operator activated="true" class="rename" compatibility="7.0.001" expanded="true" height="82" name="Rename" width="90" x="313" y="34">
<parameter key="old_name" value="%{symbol}_OPEN"/>
<parameter key="new_name" value="open"/>
<list key="rename_additional_attributes">
<parameter key="%{symbol}_DAY_HIGH" value="high"/>
<parameter key="%{symbol}_DAY_LOW" value="low"/>
<parameter key="%{symbol}_CLOSE" value="close"/>
<parameter key="Date" value="dates"/>
</list>
</operator>
<operator activated="true" class="select_attributes" compatibility="7.0.001" expanded="true" height="82" name="Select Attributes" width="90" x="447" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="dates"/>
<parameter key="invert_selection" value="true"/>
</operator>
<operator activated="true" class="write_csv" compatibility="7.0.001" expanded="true" height="82" name="Write CSV" width="90" x="581" y="34">
<parameter key="csv_file" value="C:\Users\tott_000\Dropbox\Apps\Blot\neuralmarket\public\autocharts\%{symbol}_%{end_date}_data.csv"/>
<parameter key="column_separator" value=","/>
</operator>
<operator activated="true" class="text:create_document" compatibility="7.0.000" expanded="true" height="68" name="Plot.ly" width="90" x="45" y="187">
<parameter key="text" value="<head> <!-- Plotly.js --> <script src="https://cdn.plot.ly/plotly-latest.min.js"></script> <!-- PlotlyFinance.js --> <script src="https://cdn.rawgit.com/etpinard/plotlyjs-finance/master/plotlyjs-finance.js"></script> <body> <H1>Today's Chart: %{symbol}</h1> <p>The chart below is created using the Plot.ly JS library. This a daily chart starting from %{start_date} to %{end_date}.</p> <div id="myDiv" style="width: 100%; height: 380px;"><!-- Plotly chart will be drawn inside this DIV --></div> <script> 	function makeplot() { 	Plotly.d3.csv("http://www.neuralmarkettrends.com/public/autocharts/%{symbol}_%{end_date}_data.csv", function(data){ processData(data) } ); 	}; 	function processData(allRows) { 	console.log(allRows); 	var data_open = [], data_close = [], data_high = [], data_low = [], dates = []; 	for (var i=0; i<allRows.length; i++) { 	 row = allRows[i]; data_close.push(parseFloat(row['close'])); data_high.push(parseFloat(row['high'])); data_low.push(parseFloat(row['low'])); data_open.push(parseFloat(row['open'])); 	 } 	 makePlotly( data_open, data_close, data_high, data_low ); 	} 	function makePlotly( data_open, data_close, data_high, data_low ){ 	 var data_dates = getAllDays('%{start_date}', '%{end_date}'); 	 var fig = PlotlyFinance.createCandlestick({ 	 open: data_open, 	 high: data_high, 	 low: data_low, 	 close: data_close, 	 dates: data_dates 	 }); 	fig.layout.title = 'Daily Stock Chart'; 	 fig.layout.annotations = [ 	 { 	 text: "%{symbol} Stock", 	 x: '-0.05', 	 y: 0.5, 	 xref: 'paper', 	 yref: 'paper', 	 font:{ 	 size: 18 	 }, 	 showarrow: false, 	 xanchor: 'right', 	 textangle: 270 	 } 	 ]; 	Plotly.newPlot('myDiv', fig.data, fig.layout); 	}; // Utility Function to generate all days function getAllDays(start, end) { var s = new Date(start); var e = new Date(end); var a = []; while(s < e) { a.push(s); s = new Date(s.setDate( s.getDate() + 1 )) } return a; }; makeplot(); </script> </body> "/>
</operator>
<operator activated="true" class="text:write_document" compatibility="7.0.000" expanded="true" height="82" name="Convert format" width="90" x="179" y="187"/>
<operator activated="true" class="write_file" compatibility="7.0.001" expanded="true" height="68" name="Write to HTML format (3)" width="90" x="313" y="187">
<parameter key="filename" value="C:\Users\tott_000\Dropbox\Apps\Blot\neuralmarket\2016\%{end_date}-%{symbol}-chart.html"/>
</operator>
<connect from_op="Yahoo Historical Stock Data" from_port="example set" to_op="Sort" to_port="example set input"/>
<connect from_op="Sort" from_port="example set output" to_op="Rename" to_port="example set input"/>
<connect from_op="Rename" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
<connect from_op="Select Attributes" from_port="example set output" to_op="Write CSV" to_port="input"/>
<connect from_op="Plot.ly" from_port="output" to_op="Convert format" to_port="document"/>
<connect from_op="Convert format" from_port="file" to_op="Write to HTML format (3)" to_port="file"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
</process>
</operator>
</process>
While this works great, it does have one snag. If you want to display multiple stock charts, you'd have to figure out a way to update the location of the chart relative to where your blog post is.
This is important if you show more than one post per page. When I generated two posts with two different stock charts, the HTML made them relative to the top of the page. In other words, the chart was on top of the other one. I'm sure this is an easy fix but something I'm not going to bother with for this tutorial!
Read more RapidMiner tutorials
RapidMiner Tutorials
A comprehensive list of RapidMiner Tutorial Videos and Code.
Thomas Ott
- Nope, got it to work in Markdown with the help of https://twitter.com/lllIIlIlIl ↩︎
Member discussion