Data Analysis & Simulation

Over the last five years, we have been adding new features to EasyFit mostly with business users in mind, but thanks to the nature of the product and the special academic pricing, it has become quite popular among the academic community: a quick search in Google reveals numerous research papers referring to EasyFit, just to name a few:
 

• “Co-evolution of Social and Affiliation Networks” (University of Maryland, USA) [link]
• “Power laws in top wealth distributions: evidence from Canada” (Brock University, Canada) [link]
• “Duration of Coherence Intervals in Electrical Brain Activity in Perceptual Organization” (RIKEN Brain Science Institute, Japan) [link]
• “Resource Management Schemes for Mobile Ad hoc Networks” (National University of Singapore, Singapore) [link]
• “Modelling the diffusion of innovation management theory using S-curves” (University of London, UK) [link]

(see the larger list of papers using EasyFit)

It is pleasing to see EasyFit helping researchers in such diverse disciplines get their job done in a more efficient way.

EasyFitXL - the distribution fitting add-in for Excel - was first introduced with the release of EasyFit 4.0 back in 2007. When designing EasyFitXL, we did a lot of research as to which Excel versions to support. At that time, the latest version of Excel was Excel 2007, which included some new useful features, such as the support for larger worksheets and multi-threaded worksheet recalculation capability. However, many customers were not rushing to upgrade to Excel 2007 because of it’s controversial Ribbon Interface, so we had to make EasyFitXL compatible with the previous version - Excel 2003.

According to some publicly available data, Excel 2002 and Excel 2000 still had a considerable user base, so we have made a decision to support these two older versions as well. As a result, EasyFitXL initially included support for Excel versions from 2000 through 2007, covering perhaps over 99% of all Excel installations in the world.

Last month Microsoft has released Excel 2010 which does not make a big difference in terms of data analysis, however, with its release we started receiving compatibility complaints from our customers, so we performed an in-depth testing and released an updated version of EasyFit (available for download).

Over the past five years, we have been selling our distribution fitting software through Plimus Inc. - the U.S. based company responsible for processing credit card orders and sending out the license keys to the customers who purchased our products. What we like about Plimus is that apart from making the ordering process secure and smooth for our users, this company is also very responsive to any queries, which is especially important when it comes to dealing with people’s money.

However, in some countries there are state regulations preventing customers (mostly government organizations and academic institutions) from ordering software online. To make our distribution fitting products available to users in those countries, we are in the process of creating a network of international distributors. Specifically, this year we have partnered up with a Chinese, a Mexican, and three Taiwanese software resellers who now offer our products through their local distribution channels.

Recently a customer has contacted us and noted that the Inverse Cumulative Distribution Function (the Quantile Function) of the Inverse Gaussian distribution implemented in EasyFit works well for lambda=1902.1, mu=41857.0 and P=0.9, but fails for the same lambda & mu and P=0.99. Last week we have released an updated version of EasyFit that fixes the problem, and in this post we would like to elaborate more on the issue.

Evaluating the Inverse CDF of the Inverse Gaussian Model
Since the CDF of the Inverse Gaussian distribution is quite complicated (expressed in terms of the two Laplace Integrals), the Inverse CDF of this model is not available in closed form, and cannot be easily evaluated for a given set of distribution parameters. Initially, we have implemented an iterative approximation algorithm that evaluates the ICDF(P) using the CDF as well as the PDF to speed up the calculation. The algorithm itself works very well over a great range of input parameters, however, we have placed a limitation on how many iterations it is allowed to perform.

Because EasyFit is considered an interactive data analysis tool, we are always looking for a balance between the feature set and the performance, which is especially important when using EasyFit with Excel worksheets calculated in real time. The limitation on the number of iterations is necessary to make sure the algorithm doesn’t fall into an “infinite loop”, meaning the situation when it’s unable to reach the specified accuracy regardless of how long it continues to work. The problem usually happens when we are hitting the precision limitations of the computer’s CPU: in theory, the algorithm must converge in a limited number of steps, but in reality, it will just continue iterating over and over again without any accuracy improvements.

As a solution, we have made some improvements to the algorithm, making it more robust and efficient, so it now works with the same accuracy, but for a larger range of input parameters. For example, considering the parameters that initially caused the problem (lambda=1902.1 and mu=41857.0), the ICDF(P) can be evaluated for values of P up to 0.999925, which is more than enough for most statistical analysis applications.

Should You Upgrade?
Since this minor issue does not affect the accuracy of distribution fitting, you only need to upgrade if you are experiencing problems evaluating the Inverse CDF of the Inverse Gaussian distribution for P>0.9, otherwise EasyFit 5.2 will still work well for you.

The beta testing of our new product, the Simulation & Probabilistic Analysis SDK, is now over, and we want to thank our beta testers for their effort and valuable feedback. One of the most exciting things is that during the beta testing phase, we have not detected any bugs in the SDK, indicating the initial high quality of the product.

The production version of the SDK is now available for public download, so if you are a software developer and need to add distribution fitting or simulation features to your software with no hassle, feel free to download the fully functional version of the SDK and try it free for 30 days.

Another good news is that the Christmas is coming, so we decided to make a gift to software developers who are considering to purchase the SDK: until the end of December, the SDK Developer License can be ordered at a $500 discount - click here details.

The new Software Development Kit enabling you to easily add Monte Carlo simulation and distribution analysis features to your applications is now available for public beta testing - please download the free beta version and take a look at the code examples (available in several languages, including C#, VB.NET, C++, and Visual Basic for Applications).

We would be glad to receive any feedback, questions or suggestions from you, so please feel free to drop us a line and let us know what you think about this product and how we can make it better.

Earlier in December 2008, we announced our plans to create the Software Development Kit that would allow software developers to integrate the simulation and probabilistic analysis functionality of EasyFit into their custom software solutions. Since then, we have been working hard to bring the SDK to market. Despite some delays, we are glad to announce that we have successfully completed the development, and this month we plan to release the beta version of the product.

Become a Beta Tester Today!
Taking part in our beta testing program would not just grant you access to the cutting edge software technology - the most active beta testers will also be able to purchase the SDK at an introductory price before the production version of the SDK is released. To sign up, please contact us with your specific requirements (let us know how you plan to use the SDK and what development platform (C#, VB.NET etc.) you are using.)

We have just released EasyFit Version 5.1 - the update that fixes a bug causing an incorrect calculation of the chi-squared GOF statistic for small sample sizes. To upgrade, uninstall EasyFit 5.0 from your computer, then download and install the latest version.

On April 17 2005, the Millstone nuclear generating plant in Connecticut shut down when a circuit board monitoring a steam pressure line short-circuited. “Tin whiskers” - microscopic growths of the metal from soldering points on a circuit board - were blamed for causing the problem. These whiskers are comprised of nearly pure tin, and are therefore electrically conductive.

Field failures attributable to tin whiskers have cost individual programs many millions of dollars each. As a result, manufacturers of high-reliability systems are forced to use Monte Carlo simulation models to decide whether the use of tin poses an acceptable risk in a given application.

Recently a group of NASA scientists lead by Karim J. Courey, a Principal Engineer with the Orbiter Sustaining Engineering Office, Lyndon B. Johnson Space Center, used our distribution fitting software EasyFit to better understand the underlying process and develop a probability model that can be used to improve existing Monte Carlo risk simulation models… read the full case study

There are many probability distributions developed by statisticians to model random data of different kinds, ranging from business data, finance data (stock prices) to engineering data (system failures) and environmental data (max. flood flows). While the standard Excel package includes some basic statistical functions, its support for probability distributions is very limited and almost useless for real world modelling applications. This article discusses the worksheet functions provided by EasyFitXL, the distribution fitting add-in for Excel that can be applied to perform a range of decision-making calculations using a variety of probability distributions… read the full article