Can I do this?

**questionortwo** · 02-18-2011, 05:52 PM

I want to build a predictive model for sports betting purposes but I don't know where to begin. I'm pretty sure I'll need a data scraper which I think can be built in java. But how would I get that data scraper to constantly search over and over? I guess I'm just not sure how java works AT ALL.

Without asking all of the little details upfront, can I use java to build a program that can constantly import relevant data and use factors that I specify to predict game outcomes?

More tedious details:
-can the program have sound alerts when certain criterion are met?
-can the program open the Internet browser and log into a certain website and perform tasks when certain criterion are met?

Most importantly, can a total beginner learn to do all of these things in a relatively short period of time?

**Fubarable** · 02-18-2011, 05:54 PM

Originally Posted by questionortwo

.... Most importantly, can a total beginner learn to do all of these things in a relatively short period of time?

My 2 cents: probably not, unless you find some pre-made libraries that do most of the work for you, but AI is high-level stuff and may require complex models such as perhaps neural networks.

**questionortwo** · 02-18-2011, 06:19 PM

Originally Posted by Fubarable

My 2 cents: probably not, unless you find some pre-made libraries that do most of the work for you, but AI is high-level stuff and may require complex models such as perhaps neural networks.

Can you explain that part?

**JosAH** · 02-18-2011, 06:42 PM

Originally Posted by questionortwo

Can you explain that part?

Maybe I can (partly) answer that question: Fubarable was probably talking about your predictive model while you where only talking about frillies such as beeps and other visibile stuff.

kind regards,

Jos

**quad64bit** · 02-19-2011, 02:12 AM

Most importantly, can a total beginner learn to do all of these things in a relatively short period of time?

Sure if 6 years of computer science, data structures, networking and network programming, algorithms (and analysis), neural networks, advanced prob and stat, and oh yeah, all of that while learning java is a 'short period of time'.

Considering the amount of fundamental information you need to just to start work, I'd say 4-6 years would be pretty quick.

Abstruse Goose » Prerequisites
I suggest you look at that comic to see what I am talking about - make sure to click on each picture as they pop up.

**Fubarable** · 02-19-2011, 03:53 AM

Originally Posted by JosAH

Maybe I can (partly) answer that question: Fubarable was probably talking about your predictive model while you where only talking about frillies such as beeps and other visibile stuff.

It means that I know squat about predictive models and was spewing the first bit of dreck that came into my head. I think I'll first learn String Theory though as it seems to be pretty basic stuff. Thank you Quad for the heads up!

**questionortwo** · 02-19-2011, 03:49 PM

So you don't think I could learn the things specific to my project in a relatively short time? It's not like I have to master java. I just need to build some good data scrapers and do statistical analysis on said data(which I guess could be done in mysql). So it's possible that all I need to learn is SQL and how to build some data scrapers. Am I missing something?

**rp181** · 02-19-2011, 04:28 PM

Gathering data is not the hard part, analyzing it is. Do you have any idea how to use this info? I would say it is almost impossible to do with out neural networks in conjunction with scraped data. The data would have to be pre-structured.

Consider Watson. The problem wasn't getting the info, it was using it.

**JosAH** · 02-19-2011, 04:49 PM

Originally Posted by rp181

Gathering data is not the hard part, analyzing it is. Do you have any idea how to use this info? I would say it is almost impossible to do with out neural networks in conjunction with scraped data. The data would have to be pre-structured.

Neural networks aren't the major part of the problem either: knowing how to model what data is crucial here. If you e.g. want to predict the lottery outcome every bet is as good as another; it you want to predict an outcome of a match historic information (previous outcomes of the matches between those two teams) are crucial. Neural nets are just silly matrixes and weight factors and a clear sign that we don't know how to solve a particular problem in an analytical way ;-)

kind regards,

Jos

**quad64bit** · 02-20-2011, 01:55 AM

Yeah even scraping isn't that simple. You usually need to know about DOM, XML, regex, Sockets, Streams, etc, then SQL queries, database design, normalization, analysis, statistical models, etc... If you already understand these concepts in another language but just don't know java, then it would be possible to do in a much shorter time (maybe even weeks). But if the programming and math are new to you, then, I'd say many months if you did nothing but, years if you also have a life.

**quad64bit** · 02-20-2011, 01:56 AM

I think I'll first learn String Theory though as it seems to be pretty basic stuff. Thank you Quad for the heads up!]

Yeah no problem. Light reading :)

**pbrockway2** · 02-20-2011, 03:08 AM

Neural nets are just silly matrixes and weight factors and a clear sign that we don't know how to solve a particular problem in an analytical way ;-)

Indeed. AFAIK sports teams still resort to actually playing the games for want of any more reliable, analytic, solution to the problem of who will win.

@OP: you could halve the problem by deciding on EITHER gathering data OR analysing it. STill the implementation might not be feasible in a short period of time unless you are already familiar with Java or some other programming language. (Ie learning the language is a third task.) I would rate the task of making a GUI beep and flash attractively (or even just be useable) as yet another project.

Recently I undertook the task of taking a whole bunch of head and shoulder photographs where there was no uniformity of contrast or position and size of the face and turning them into something uniform. I hacked something together - but the thought struck me that - if I had time - it might be fun to do this properly and implement one of the standard techniques for the core problem here: is a given pixel part of the face or not.

**Fubarable** · 02-20-2011, 04:48 AM

Originally Posted by pbrockway2

Indeed. AFAIK sports teams still resort to actually playing the games for want of any more reliable, analytic, solution to the problem of who will win.

Gawd, I pooped in my pants with this one. Too funny!

**toadaly** · 02-20-2011, 05:51 AM

Originally Posted by questionortwo

I want to build a predictive model for sports betting purposes but I don't know where to begin. I'm pretty sure I'll need a data scraper which I think can be built in java. But how would I get that data scraper to constantly search over and over? I guess I'm just not sure how java works AT ALL.

Yes, you can build a data scraper in Java.

Without asking all of the little details upfront, can I use java to build a program that can constantly import relevant data and use factors that I specify to predict game outcomes?

Yes.

More tedious details:
-can the program have sound alerts when certain criterion are met?
-can the program open the Internet browser and log into a certain website and perform tasks when certain criterion are met?

Yes.

Most importantly, can a total beginner learn to do all of these things in a relatively short period of time?

Probably not, unless you're an incredibly quick learner. If you are a whiz at predictive algorithms, I suggest teaming up with a good programmer to do the rest. You might be able to save some development time by purchasing the data scraper instead of building it.