Close

In this tutorial I am going to design a visualisation of stock price correlations in Stator using the D3.js visualisation library. This visual study will be highly interactive allowing you to view correlations at any point in time along with the ability to view the complete correlation path for the time period being analysed.

So let’s get to it-

### What do I need?

You need the latest version of Stator which includes visual studies functionality, if you haven’t already installed Stator then visit the download page to get started, it’s an easy installation. You will also need access to security price data which you can connect to using one of our many (free) data plugins.

For those of you that are unfamiliar with Stator, it’s leading portfolio management software designed for the financial markets and used by professional traders, retail traders and hedge funds all around the world. D3, which is the visualisation library we’ll be using is a leading JavaScript library that supports dynamic and interactive data visualisations. These two technologies combined enable you to analyse financial data in ways never before imagined– all via functionality known as “Visual Studies”.

### Correlation, what is it?

In the world of finance, a correlation is a statistical measure of how two securities move in relation to one another. Correlation is computed into what’s known as a “correlation coefficient” which ranges between -1 and 1. A correlation coefficient of 1 simply means as one security moves up or down the other will move in exactly the same direction. A correlation coefficient of -1 means as one security moves up or down the other will move in the opposite direction. A value of 0 means the two securities more without any relation to one another, they are said to move randomly.

As a general rule:

• Coefficient = 1 = same movement in prices observed
• Coefficient = 0 = no relationship between price movement
• Coefficient = -1 = opposite movement in prices observed

Calculating the correlation coefficient

For the purposes of this visualisation I am going to calculate the “Pearson’s Correlation” between each of the selected securities. “Pearson’s Correlation” benchmarks the linear relationship whilst another commonly used correlation “Spearman’s” benchmarks monotonic relationship. As this visualisation looks at the linear relationship between security prices we will use “Pearson’s Correlation”.

Calculating the correlation manually isn’t difficult, just tedious if the data set is large. Later in this tutorial we’ll code a generic correlation function to handle the computation.

The formula to calculate the “Pearson’s Correlation Coefficient” is:

x = security 1 average closing price
y = security 2 average closing price

To calculate the “Pearson’s Correlation Coefficient” in a single pass you can use the following formula.

For the purposes of this visualisation, the “Pearson’s Correlation Coefficient” will be calculated using a moving average of the closing price. This will provide a more stable correlation over time as the primary purpose is to identify trends in the movement of correlation.

The steps to calculate the “Pearson’s Correlation” for two securities in our visualisation is as follows:

1. Calculate the mean (average) of x and y
2. Subtract the mean of x from every x value (call it “a“), the same for y (call it “b“)
3. Calculate a x ba2 and b2 for every value
4. Sum a x b, sum a2, sum b2
5. Divide the sum of a x b by the square root of ((sum of a2) x (sum of b2))

Are you still with me?

This calculation will result in a value between -1 and 1 and repeating this calculation for each date in the data period will generate a correlation-time series which we can plot in addition to a single correlation calculation. This will become more evident as we advance further into the tutorial.

### What key design elements do we want for this visualisation?

One key feature of this visualisation will be “interactivity”. Allowing the user to observe changes in correlation over time will help to tell a story and hopefully it will highlight trends which may influence future investment decisions. Ideally, we’d like to make the process of viewing correlation changes over time as user-friendly as possible which can achieve using a simple “mouseover” effect as shown in the image below:

As the user moves the mouse cursor from left to right over the date area, the underlying date will change forcing the visualisation to refresh and thus updating the plot of security correlations.

Another feature we’ll develop for this visualisation is the ability to view an entire correlation path (correlation-time series). We’ll allow the user to toggle path visibility for all correlations or just a single correlation. The image shown below illustrates how a single security pair correlation is overlaid onto the visualisation as the mouse cursor is positioned over the security correlation circle.

So what exactly are we seeing in the picture above?

It’s the correlation over time of two securities, GE and Apple Inc. The end date of this analysis is the 2nd of March 2018 and we can clearly see how the correlation has changed from near 1.0 at the start of the study (June 2013) to -0.0493 in March 2018. It’s telling us that GE and Apple Inc are no longer correlated- but they once were!

Another important feature that needs to be mentioned here is the ability to calculate and plot correlations for multiple securities. What we want to do is allow the user to select multiple (any number of) securities using their mouse, all of which will have correlations calculated and plotted. The visualisation will determine all the possible security pairings and will calculate and display correlations accordingly. Sounds impressive doesn’t it?

It’s also important to note that any visual studies which you write for Stator should be dynamic in their ability to handle data with differing ranges. That is, the data accessed by a visual study will maintain the same format with different ranges. For this reason a visual study needs to dynamically recalculate scales each time the study is refreshed or new data added.

### Cleansing the underlying data

Data for this visual studies ‘data domain’ is in the following JSON format:

```
Javascript object notation (JSON)
- Each data point is represented by the following JSON.
- Date is in yyyy-MM-dd HH:mm:ss format (HH = 24 hour notation).

[
{
"DateTime": "2013-01-02 13:00:00",
"Open": 1.456585,
"High": 1.482289,
"Low": 1.430881,
"Close": 1.473721,
"Volume": 195979,
"OpenInterest": 0,
"Change": 0,
"Identifier": null,
"Misc": null,
"ErrorMessage": null
}
]```

Immediately we see the “DateTime” field is represented by a string which isn’t code friendly. Ideally, we need to convert this date field into a JavaScript “Date” object which we accomplish using the code snippet shown below. The code cycles through each date in the JSON file and parses the date portion into a JavaScript “Date” object. Extra processing is performed to determine the minimum and maximum dates in the data file which are used to dynamically calculate scales etc.

```
// The JSON data data needs to be parsed into a ds.js date object.
var parseDate = d3.timeParse("%Y-%m-%d");

// JSON Data processing
// This only has to be performed once for the study
// 1. Trimming to remove empty datasets
// 2. Date cleaning
cleanJSON();
function cleanJSON() {
var cleanDataPrice = [];
for (var i = 0; i < dataPrice.length; i++) {
var price = dataPrice[i];
if (price.length == 0) { continue;}

price.forEach(function(d) {
var datePart = d.DateTime.substring(0,10);
d.DateTime = parseDate(datePart);
});

var min = d3.min(price, function(d) { return d.DateTime; });
var max = d3.max(price, function(d) { return d.DateTime; });
if (min < minDate) minDate = min;
if (max > maxDate) maxDate = max;

cleanDataPrice.push(dataPrice[i]);
}

dataPrice = cleanDataPrice;
}```

### Utility functions

This visualisation relies on a lot of calculations being performed on the underlying data. To avoid code duplication we’ve written a few utility functions/methods which have single or limited scope of responsibility. Code reuse is an important aspect of smart programming and shown below are a few of the utility functions which are used in this visualisation.

A date in plain English

You’ve may have noticed the nice English representation of date in this visualisation, that is, the 1/1/2018 is written as “1st January 2018”. This is achieved using the utility function shown below. It takes a JavaScript “Date” object as an input parameter and returns an English string representation of the date.

```
// Date formatting function
function GetDateInEnglish(date) {
var m_names = new Array("January", "February", "March",
"April", "May", "June", "July",
"August", "September", "October",
"November", "December");

var d = date;
var curr_date = d.getDate();
var sup = "";
if (curr_date == 1 || curr_date == 21 || curr_date == 31) {
sup = "st";
}
else if (curr_date == 2 || curr_date == 22) {
sup = "nd";
}
else if (curr_date == 3 || curr_date == 23) {
sup = "rd";
}
else {
sup = "th";
}

var curr_month = d.getMonth();
var curr_year = d.getFullYear();

return curr_date + sup + " " + m_names[curr_month] + " " + curr_year;
}```

Calculating correlation coefficient

Credit to Steve Gardner for writing this correlation coefficient function in JavaScript. The function takes two array input parameters and returns the Pearson’s Correlation Coefficient. You’ll notice the function has a ‘protective’ “if” clause to warn the user if the two input arrays are not of the same length (size). As you’ll see by reviewing the full visualisation code we ensure the two input arrays have the same size before being passed to the correlation function.

```// Calculate the pearsons correlation for two passed in arrays
// Credit to Steve Gardner for this function
// http://stevegardner.net/2012/06/11/javascript-code-to-calculate-the-pearson-correlation-coefficient/
function getPearsonsCorrelation(x, y) {
var shortestArrayLength = 0;
if(x.length == y.length)
{
shortestArrayLength = x.length;
}
else if(x.length > y.length)
{
shortestArrayLength = y.length;
console.error('x has more items in it, the last ' + (x.length - shortestArrayLength) + ' item(s) will be ignored');
}
else
{
shortestArrayLength = x.length;
console.error('y has more items in it, the last ' + (y.length - shortestArrayLength) + ' item(s) will be ignored');
}

var xy = [];
var x2 = [];
var y2 = [];

for(var i=0; i<shortestArrayLength; i++)
{
xy.push(x[i] * y[i]);
x2.push(x[i] * x[i]);
y2.push(y[i] * y[i]);
}

var sum_x = 0;
var sum_y = 0;
var sum_xy = 0;
var sum_x2 = 0;
var sum_y2 = 0;

for(var i=0; i<shortestArrayLength; i++)
{
sum_x += x[i];
sum_y += y[i];
sum_xy += xy[i];
sum_x2 += x2[i];
sum_y2 += y2[i];
}

var step1 = (shortestArrayLength * sum_xy) - (sum_x * sum_y);
var step2 = (shortestArrayLength * sum_x2) - (sum_x * sum_x);
var step3 = (shortestArrayLength * sum_y2) - (sum_y * sum_y);
var step4 = Math.sqrt(step2 * step3);
var answer = step1 / step4;

}```

Calculating an average (the mean)

The following utility function takes an array as an input parameter and returns the calculated mean.

```
// Calculate an average
function mean(arr) {
var i,
sum = 0,
len = arr.length;
for (i = 0; i < len; i++) {
sum += arr[i];
}
var avg = sum / len;
if (isNaN(avg)) return 0;
return avg;
}```

### Summary

So at this point you’re probably interested in seeing all the code working together. Without further hesitation please find the complete visual study “Correlation vs Turnover” shown below. This visual study is included when you download Stator and you’re free to edit and change the code to your own needs. Included with Stator is a fully integrated development environment (IDE) which you can use to test, write code and play with visual studies.

```
// ENVIRONMENT SETUP
// windowWidth and windowHeight are set in the HEAD section using jQuery.
// This ensures the chart is sized to fill the available space.
var margin = {top: 20, right: 10, bottom: 20, left: 25},
width = windowWidth - margin.left - margin.right,
height = windowHeight - margin.top - margin.bottom,

// MISC. FUNCTIONS
// The JSON data data needs to be parsed into a ds.js date object.
var parseDate = d3.timeParse("%Y-%m-%d");
var minDate = new Date(9999, 01, 01),
maxDate = new Date(0000, 01, 01),
nFormat = d3.format(",.0f"),
colourScale = d3.scaleOrdinal(d3.schemeCategory20);

// JSON Data processing
// This only has to be performed once for the study
// 1. Trimming to remove empty datasets
// 2. Date cleaning
cleanJSON();
function cleanJSON() {
var cleanDataPrice = [];
for (var i = 0; i < dataPrice.length; i++) {
var price = dataPrice[i];
if (price.length == 0) { continue;}

price.forEach(function(d) {
var datePart = d.DateTime.substring(0,10);
d.DateTime = parseDate(datePart);
});

var min = d3.min(price, function(d) { return d.DateTime; });
var max = d3.max(price, function(d) { return d.DateTime; });
if (min < minDate) minDate = min;
if (max > maxDate) maxDate = max;

cleanDataPrice.push(dataPrice[i]);
}

dataPrice = cleanDataPrice;
}

// Analysis variables
var securityData,
periodLookback = 10,
fromDate = minDate,
longestPriceSeries = {},
securityCorrelationData = [],
correlationToDate = maxDate,
correlationData = {},
orderedDateReference = [],
minCombinedTO = Number.MAX_VALUE,
maxCombinedTO = 0,
minACP = Number.MAX_VALUE,
maxACP = 0;

// Data in dataPrice[] contains price data for the selected securities
// We need to perform the following actions on the data
// 	1. Filter the data by date
//	2. Calculate the average closing price for the periodLookback
//	3. Calculate the avarage volume for the periodLookback
// 	4. Calculate the average (from 1 and 2) turnover
securityData = processJSON(fromDate, correlationToDate);
function processJSON(fromDate, processToDate) {

if (processToDate == null) processToDate = maxDate;
if (fromDate == null) fromDate = minDate;

var securityData = [];

longestPriceSeries.length = 0;
longestPriceSeries.key = "";

// Loop through each security
for (var i = 0; i < dataPrice.length; i++) {

// Extract the data falling between the two dates
var data = Enumerable.From(dataPrice[i])
.Where(function(x) {
return x.DateTime >= fromDate && x.DateTime <= processToDate;
})
.Select(function(x) {
return {
close: x.Close,
volume: x.Volume,
datetime: x.DateTime,
identifier: x.Identifier
}
})
.ToArray();

// data variable contains data for a security filtered
// by date (the fromDate and the processToDate)
// Cylce through the data, calculating the moving
// average for closing price and volume
// We also add the turnover to the array (avgclose x avgvolume)
var identifier;
var security = {};
for (var s = periodLookback-1; s < data.length; s++) {
var iml = s-periodLookback;
var p = data.slice(iml, s);
closearr = [];
volumearr = [];
for (var x = 0; x < p.length; x++) {
closearr.push(p[x].close);
volumearr.push(p[x].volume);
}

data[s].avgclose = mean(closearr);
data[s].avgvolume = mean(volumearr);
data[s].avgturnover = data[s].avgclose * data[s].avgvolume;

identifier = data[s].identifier;
var key = getSecurityKey(data[s]);
security[key] = data[s];
}

// Determine the longest price series
if (longestPriceSeries.length < data.length) {
longestPriceSeries.length = data.length;
longestPriceSeries.key = identifier;
}

// Colour for the security
security.colour = colourScale(i);

// Calculate combinations of securities to be
// used later in the calculation of correlation
findCombinations(identifier, dataPrice, i);

// Add the security data the return object
securityData[identifier] = security;
}

return securityData;
}

// The securityData needs to be padded with dates so that each object is equal length
var closeArrays = {};
var baseSecurity = securityData[longestPriceSeries.key];
var createdDateReference = false;
for (var key in securityData) {
var security = securityData[key];
var closeArray = [];
for (d in baseSecurity) {
if (!createdDateReference) orderedDateReference.push(new Date(d));
if (security[d] == null) {
security[d] = getEmptyDayData(key, d);
}
var closePrice = security[d].avgclose;
if (isNaN(closePrice)) closePrice = 0;
closeArray.push(closePrice);
}

closeArrays[key] = closeArray;
createdDateReference = true;
}
}

// With security data calculated (averages etc) we need to create a matrix of
// security correlations for a passed in date, this will become the plotted data
calculateSecurityCorrelations();
function calculateSecurityCorrelations() {
var baseSecurity = securityData[longestPriceSeries.key];
// We calculate correlations up to the calculate date
// Cycle through each of the security combinations
for (var key in securityCorrelationData) {
var c = securityCorrelationData[key];

var i = 1;
for (d in baseSecurity) {
var cd = {};

var security1 = closeArrays[c.id1].slice(0, i);
var security2 = closeArrays[c.id2].slice(0, i);

// Colour and data point count
cd.datapointcount = security1.length;
cd.colour1 = securityData[c.id1].colour;
cd.colour2 = securityData[c.id2].colour;

// Correlation calculation
var correlation = getPearsonsCorrelation(security1, security2);
cd.correlation = correlation;
cd.dateTO = d;
cd.dateFROM = minDate;
cd.id1 = c.id1;
cd.id2 = c.id2;
cd.identifier = c.identifier;

// Average closing price
var ACP1 = securityData[c.id1][d].avgclose;
var ACP2 = securityData[c.id2][d].avgclose;
cd.acp1 = ACP1;
cd.acp2 = ACP2;
cd.avgclosingprice = (ACP1 + ACP2)/2;
if (cd.avgclosingprice < minACP) minACP = cd.avgclosingprice;
if (cd.avgclosingprice > maxACP) maxACP = cd.avgclosingprice;

// Turnover calculation
var TO1 = securityData[c.id1][d].avgturnover;
var TO2 = securityData[c.id2][d].avgturnover;
cd.to1 = TO1;
cd.to2 = TO2;
cd.totalturnover = TO1 + TO2;
if (cd.totalturnover < minCombinedTO) minCombinedTO = cd.totalturnover;
if (cd.totalturnover > maxCombinedTO) maxCombinedTO = cd.totalturnover;

c[d] = cd;
i += 1;
}
}
}

// Convert the security correlation data into a plottable object
// for the passed in analysis date
var correlationToPlot = getPlottableDataForDate(correlationToDate);
function getPlottableDataForDate(correlationDate) {
// Get the data out of the correlation plot object for the date
var forDateData = [];
for (c in securityCorrelationData) {
var dataForDate = securityCorrelationData[c][correlationDate];
forDateData.push(dataForDate);
}
return forDateData;
}

// Convert the security correlation data to produce tails for the circles
// from the start date to the selected date. This is used to display
// tails for each of the circles if required
function plotTails(eraseAllTailsAndExit, pair) {

var tails = svg.selectAll(".tail");
tails.remove();
if (eraseAllTailsAndExit) {
return;
}

var tailsData = [];
for (key in securityCorrelationData) {
if (pair != null && pair != key) continue;
var securityData = [];
var c = securityCorrelationData[key];
for (var i = 1; i < orderedDateReference.length-1; i++) {
var d = orderedDateReference[i];
var cd = c[d];
var data = {};
data.x = x_scale(cd.totalturnover);
data.y = y_scale(cd.correlation);
data.colour = cd.colour1;
securityData.push(data);
}
tailsData.push(securityData);
}

//console.log(tailsData);

// Draw all the tails
var line = d3.line()
.curve(d3.curveBasis)
.x(function(d) { return d.x; })
.y(function(d) { return d.y; });

svg.selectAll(".tail")
.data(tailsData)
.enter()
.append("path")
.attr("class", "tail")
.style("stroke", function(d, i) { return d[i].colour; })
.attr("d", line);
}

//console.log(securityCorrelationData);

// UTILITY. FUNCTIONS
function getClosestDateOrPrevious(forDate) {
var useDate = maxDate;
var dateList = orderedDateReference;
for (var i = dateList.length-1; i >= 0; i--) {
if (dateList[i] <= forDate) {
useDate = dateList[i];
break;
}
}
return useDate;
}

// Get array from object property
function getArrayFromObjectProperty(data) {
var sortable = [];
for (var key in data) {
sortable.push([vehicle, maxSpeed[vehicle]])
}
sortable.sort(function(a, b) {return a[1] - b[1]})
}

// Get empty day data
function getEmptyDayData(identifier, forDate) {
var day = {};
day.datetime = new Date(forDate);
day.close = 0;
day.volume = 0;
day.avgclose = 0;
day.avgvolume = 0;
day.avgturnover = 0;
day.identifier = identifier;
return day;
}

// Get the security key
function getSecurityKey(data) {
return data.datetime;
}

// Calculate an average
function mean(arr) {
var i,
sum = 0,
len = arr.length;
for (i = 0; i < len; i++) {
sum += arr[i];
}
var avg = sum / len;
if (isNaN(avg)) return 0;
return avg;
}

// Calculate combinations of securities
function findCombinations(id, data, i) {
for (var x = i; x < data.length; x++) {
var sid = data[x][0].Identifier;
if (id != sid) {
var c = {};
c.identifier = id + "-" + sid;
c.id1 = id;
c.id2 = sid;
securityCorrelationData[c.identifier] = c;
}
}
}

// Object size extension method
Object.size = function(obj) {
var size = 0, key;
for (key in obj) {
if (obj.hasOwnProperty(key)) size++;
}
return size;
};

// Print each security size to the console
// This is a test function, called for sanity
function printSecuritySize() {
for (var key in securityData) {
var size = Object.size(securityData[key]);
console.log(key + " " + size);
}
}

// Converts an epoch number into a date object
function getDateFromNumber(dateAsNumber) {
var date = new Date(dateAsNumber);
return date;
}

// Return the text used to indicate the data points
function getDataPointText() {
return correlationToPlot[0].datapointcount + " data points used in calculation";
}

// Date formatting function
function GetDateInEnglish(date) {
var m_names = new Array("January", "February", "March",
"April", "May", "June", "July",
"August", "September", "October",
"November", "December");

var d = date;
var curr_date = d.getDate();
var sup = "";
if (curr_date == 1 || curr_date == 21 || curr_date == 31) {
sup = "st";
}
else if (curr_date == 2 || curr_date == 22) {
sup = "nd";
}
else if (curr_date == 3 || curr_date == 23) {
sup = "rd";
}
else {
sup = "th";
}

var curr_month = d.getMonth();
var curr_year = d.getFullYear();

return curr_date + sup + " " + m_names[curr_month] + " " + curr_year;
}

// Calculate the pearsons correlation for two passed in arrays
// Credit to Steve Gardner for this function
// http://stevegardner.net/2012/06/11/javascript-code-to-calculate-the-pearson-correlation-coefficient/
function getPearsonsCorrelation(x, y) {
var shortestArrayLength = 0;
if(x.length == y.length)
{
shortestArrayLength = x.length;
}
else if(x.length > y.length)
{
shortestArrayLength = y.length;
console.error('x has more items in it, the last ' + (x.length - shortestArrayLength) + ' item(s) will be ignored');
}
else
{
shortestArrayLength = x.length;
console.error('y has more items in it, the last ' + (y.length - shortestArrayLength) + ' item(s) will be ignored');
}

var xy = [];
var x2 = [];
var y2 = [];

for(var i=0; i<shortestArrayLength; i++)
{
xy.push(x[i] * y[i]);
x2.push(x[i] * x[i]);
y2.push(y[i] * y[i]);
}

var sum_x = 0;
var sum_y = 0;
var sum_xy = 0;
var sum_x2 = 0;
var sum_y2 = 0;

for(var i=0; i<shortestArrayLength; i++)
{
sum_x += x[i];
sum_y += y[i];
sum_xy += xy[i];
sum_x2 += x2[i];
sum_y2 += y2[i];
}

var step1 = (shortestArrayLength * sum_xy) - (sum_x * sum_y);
var step2 = (shortestArrayLength * sum_x2) - (sum_x * sum_x);
var step3 = (shortestArrayLength * sum_y2) - (sum_y * sum_y);
var step4 = Math.sqrt(step2 * step3);
var answer = step1 / step4;

}

// Various accessors that specify the four dimensions of data to visualize.
function x(d) { return d.totalturnover; }
function y(d) { return d.correlation; }
function key(d) { return d.identifier; }
function radius(d) { return d.avgclosingprice; }

var bisect = d3.bisector(function(d) { return d[0]; });

// CHART SETUP
// Scaling
var x_scale, y_scale, r_scale;
setScaling();
function setScaling() {
var nodes = correlationToPlot;

var x_extent = d3.extent(nodes, function(d) { return x(d); })
var x_max = Math.max(Math.abs(x_extent[1]));
x_scale = d3.scaleLinear()
.domain([minCombinedTO*0.80, maxCombinedTO*1.06])
.range([margin.left, width]);

y_scale = d3.scaleLinear()
.domain([1, -1])
.range([margin.top, height]);

r_scale = d3.scaleSqrt()
.domain([minACP, maxACP])
.range([height/50, height/20]);
}

// The x & y axes.
var xAxis = d3.axisBottom(x_scale).tickArguments(5).tickFormat(d3.format("0s")),
yAxis = d3.axisLeft(y_scale);

// Setup the SVG panel for display of the chart.
var svg = d3.select("body")
.append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")")
.append("g");

if (correlationToPlot.length > 0) {
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
}

var yAxisLeft = margin.left - 10;
svg.append("g")
.attr("class", "y axis")
.attr("transform", "translate(" + yAxisLeft + ", 0)")
.call(yAxis);

svg.append("text")
.attr("class", "x label")
.attr("text-anchor", "end")
.attr("x", width)
.attr("y", height - 6)
.text("Average (Combined) Turnover");

svg.append("text")
.attr("class", "y label")
.attr("text-anchor", "end")
.attr("y", yAxisLeft + 5)
.attr("x", (height * -1) + 70)
.attr("dy", ".75em")
.attr("transform", "rotate(-90)")
.text("Correlation");

// Add the date label; the value is set on transition.
var label = svg.append("text")
.attr("class", "date label")
.attr("text-anchor", "end")
.attr("y", height - 30)
.attr("x", width)
.text(GetDateInEnglish(maxDate));

// Append instructions
var bottom = height + margin.bottom + 5;
var instructions = svg.append("text")
.attr("class", "instructions")
.attr("text-anchor", "end")
.attr("x", width)
.attr("y", -4)
.text(getDataPointText());

instructions.append("tspan")
.attr("x", width)
.attr("dy", "1.2em")
.text("Move mouse over date to animate");

// Draw lines to mark 0 on the y
svg.append("svg:line")
.style("stroke", "#D1D1D1")
.style("shape-rendering", "crispEdges")
.attr("x1", margin.left)
.attr("x2", width)
.attr("y1", y_scale(0))
.attr("y2", y_scale(0));

// Add an overlay for the date label.
var box = label.node().getBBox();

var overlay = svg.append("rect")
.attr("class", "overlay")
.attr("x", box.x)
.attr("y", box.y)
.attr("width", box.width)
.attr("height", box.height)
.on("mouseover", enableInteraction);

// Legend
var names = [];
for (s in securityData) {
var d = {};
d.colour = securityData[s].colour;
d.identifier = s;
names.push(d);
}
var bottomOfLegend = height - 105;
var legend = svg.append("g")
.attr("transform", "translate(0, " + bottomOfLegend + ")")
.selectAll(".legend")
.data(names)
.enter().append("g")
.attr("class", "legend")
.attr("transform", function (d, i) {
return "translate(0," + -i * 15 + ")";
});

legend.append("rect")
.attr("x", width - 10)
.attr("width", 10)
.attr("height", 10)
.style("fill", function (d) { return d.colour; });

legend.append("text")
.attr("x", width-14)
.attr("y", 5)
.attr("dy", ".35em")
.style("text-anchor", "end")
.text(function (d) { return d.identifier; });

// Add each dot to represent each security correlation
var dot;
drawDots();
function drawDots() {
dot = svg.selectAll(".dot")
.data(correlationToPlot, function(d){ return d.identifier; });
dot.exit()
.remove();
dot.transition()
.duration(100)
.call(position);
dot.sort(order);
dot.enter()
.append("circle")
.attr("class", "dot")
.style("fill", function(d){
colourCircles(d.identifier, d.colour1, d.colour2)
return "url(#grad" + d.identifier + ")"
})
.call(position)
.sort(order)
.on("mouseover", mouseOver)
.on("mouseout", mouseOut);
}

dot.append("title")
.text(function(d) { return d.identifier; });

// Positions the dots based on data.
function position(dot) {
dot.attr("cx", function(d) { return x_scale(x(d)); })
.attr("cy", function(d) { return y_scale(y(d)); })
.attr("r", function(d) { return r_scale(radius(d)); });
}

// Defines a sort order so that the smallest dots are drawn on top.
function order(a, b) {
}

// Colour for each circle
function colourCircles(id, colour1, colour2){
.attr("x1", "100%").attr("x2", "0%").attr("y1", "0%").attr("y2", "0%");
}

enableInteraction();
function enableInteraction() {
var dateScale = d3.scaleLinear()
.domain([minDate, maxDate])
.range([box.x + 10, box.x + box.width - 10])
.clamp(true);

// Cancel the current transition, if any.
svg.transition().duration(0);

overlay
.on("mouseover", mouseover)
.on("mouseout", mouseout)
.on("mousemove", mousemove)
.on("touchmove", mousemove);

function mouseover() {
label.classed("active", true);
}

function mouseout() {
label.classed("active", false);
}

function mousemove() {
var dateAsNumber = dateScale.invert(d3.mouse(this)[0]);
var dateMouse = getDateFromNumber(dateAsNumber);
var date = getClosestDateOrPrevious(dateMouse);
displayDate(date);
refreshPlottedDataForDate(date);
}
}

// Refresh the data displayed on the chart
function refreshPlottedDataForDate(date) {
if (correlationToDate == date) return;
correlationToDate = date;
correlationToPlot = getPlottableDataForDate(correlationToDate);
drawDots();
}

// Updates the display to show the specified date.
function displayDate(date) {
instructions.text(getDataPointText());
label.text(GetDateInEnglish(date));
}

// Mouse events
var selectedCorrelation;
function mouseOver(d) {
selectedCorrelation = d;
plotTails(false, d.identifier);
}

function mouseOut() {
if (!ShowingTails) {
plotTails(true);
}
else {
plotTails(false);
}
}

// Tooltip methods
function getTooltipTitle() {
return selectedCorrelation.identifier;
}

function getTooltipText() {
var c = selectedCorrelation;
return 	"For Date: <b>" +  GetDateInEnglish(correlationToDate) + "</b><br>" +
"Correlation: <b>" + c.correlation.toFixed(4) + "</b><br>" +
"Total Turnover (TO): <b>" + nFormat(c.totalturnover) + "</b><br>" +
"" + c.id1 + " TO: " + nFormat(c.to1) + "<br>" +
"" + c.id2 + " TO: " + nFormat(c.to2) + "";
}

assignTooltips();
function assignTooltips() {
\$('.dot').qtip({
overwrite: true,
content: {
title: getTooltipTitle,
text: getTooltipText
},
position: {
viewport: \$(window)
},
show: {
delay: 30,
effect: function() {
}
},
style: {
classes: 'qtip-light qtip-rounded qtooltip'
}
})
}

// Show and hide tails
var ShowingTails = false;
.attr("text-anchor", "end")
.attr("x", width)
.attr("y", height - 75)
.on("click", function(d){ showOrHideTails();});

var text = "Show Tails";
if (ShowingTails == true) {
text = "Hide Tails";
}
return text;
}

function showOrHideTails() {
plotTails(ShowingTails);
ShowingTails = !ShowingTails;