Forum update

Forum info, updates, discussion. If you have a request for something you want to see on the forum, feel free to bring it up here.
User avatar
ST_Lawson
Site Admin
Posts: 8247
Joined: Wed Apr 16, 2014 1:15 pm
Location: Macomb, IL
Contact:

sealhall74 wrote: Wed Aug 16, 2017 3:32 pm If you don't mind doing on AGS, I think that would be better as you have a lot more "rapartee" going there than I do. FYI, three game picks during the opening weekend and none are MVFC. I didnt really want to include them but my program chooses the best non-MVFC matchups and those just happened to be it so it is what it is.
And done: http://www.anygivensaturday.com/showthr ... kin-Parlay
Thank you so much for doing this again this year. It's an awesome thing for our message board and for MVFC fans.
Scott Lawson - Board Admin
Western Illinois University Alum/Fan/Employee
Member of the Marching Leathernecks - 1996-2000
User avatar
sealhall74
Posts: 5770
Joined: Fri Apr 18, 2014 1:18 pm
Location: Wherever, Windblows

ST, is it possible to add the Pickem choice to the mobile menu group?
Embrace the pace of the race.
User avatar
ST_Lawson
Site Admin
Posts: 8247
Joined: Wed Apr 16, 2014 1:15 pm
Location: Macomb, IL
Contact:

sealhall74 wrote: Wed Aug 23, 2017 1:42 pm ST, is it possible to add the Pickem choice to the mobile menu group?
I don't know, but I'll see what I can do.
Scott Lawson - Board Admin
Western Illinois University Alum/Fan/Employee
Member of the Marching Leathernecks - 1996-2000
User avatar
sealhall74
Posts: 5770
Joined: Fri Apr 18, 2014 1:18 pm
Location: Wherever, Windblows

The interest in the Pickem game has far exceeded my expectations. Trying to get decent performance is my next objective. When your data and code reside somewhere out in the cloud on a shared computer withe Lord only know who, it is difficult as your stuff is always getting flushed out of memory when the other applications kick in. These are the lessons you learn in life. If I need a dedicated Google computer, I will do it.
Embrace the pace of the race.
User avatar
ST_Lawson
Site Admin
Posts: 8247
Joined: Wed Apr 16, 2014 1:15 pm
Location: Macomb, IL
Contact:

sealhall74 wrote: Sun Aug 27, 2017 7:58 pm The interest in the Pickem game has far exceeded my expectations. Trying to get decent performance is my next objective. When your data and code reside somewhere out in the cloud on a shared computer withe Lord only know who, it is difficult as your stuff is always getting flushed out of memory when the other applications kick in. These are the lessons you learn in life. If I need a dedicated Google computer, I will do it.
Well, you did ask me to post it for you ;)

Idk how you have the backend set up, but if there's anything I can do in terms of storage space, databases, etc. just let me know and I'll see what I can do.
Scott Lawson - Board Admin
Western Illinois University Alum/Fan/Employee
Member of the Marching Leathernecks - 1996-2000
User avatar
sealhall74
Posts: 5770
Joined: Fri Apr 18, 2014 1:18 pm
Location: Wherever, Windblows

ST_Lawson wrote: Sun Aug 27, 2017 9:05 pm
sealhall74 wrote: Sun Aug 27, 2017 7:58 pm The interest in the Pickem game has far exceeded my expectations. Trying to get decent performance is my next objective. When your data and code reside somewhere out in the cloud on a shared computer withe Lord only know who, it is difficult as your stuff is always getting flushed out of memory when the other applications kick in. These are the lessons you learn in life. If I need a dedicated Google computer, I will do it.
Well, you did ask me to post it for you ;)

Idk how you have the backend set up, but if there's anything I can do in terms of storage space, databases, etc. just let me know and I'll see what I can do.
My backend is nothing more than a spreadsheet on my Google Drive and an App Script (javascript) project also out on my Google Drive which generates the html for the user interface along with inserting/updating rows in the spreadsheet based on user game picks. I am not using any of the pay services offered by Google yet. I assume that my docs and apps are located on Google's Northern Virginia regional bucket because that is close to where I live but I dont really know for sure, all I know is that they are marked "Global". They also have a central region bucket (data center) located in Council Bluffs, Iowa. If I relocate my stuff to the heartland, things might be a little faster for most people. Without a doubt, things would be much faster if I was using a real database like Mysql on the backend. That is a good longterm goal for sure. As for the application code, I would rather not rewrite it from scratch. Changing the data access calls to talk to Mysql instead of Google's sheet api would not be that difficult but also not something I can do overnight. My strategy going for now is to continue to monitor for a week or two. If I see submit times consistently over one minute, then I will start paying Google for a couple of things which give me a little more control like where my stuff is running from. Google service charges are all based on usage and I dont see this app as being that much of a resource hog. Besides, I think there would be considerably less data traffic out in the midwest. Everyone is etiher picking corn or going to football games. ;)
Embrace the pace of the race.
User avatar
ST_Lawson
Site Admin
Posts: 8247
Joined: Wed Apr 16, 2014 1:15 pm
Location: Macomb, IL
Contact:

sealhall74 wrote: Tue Aug 29, 2017 6:42 am
My backend is nothing more than a spreadsheet on my Google Drive and an App Script (javascript) project also out on my Google Drive which generates the html for the user interface along with inserting/updating rows in the spreadsheet based on user game picks. I am not using any of the pay services offered by Google yet. I assume that my docs and apps are located on Google's Northern Virginia regional bucket because that is close to where I live but I dont really know for sure, all I know is that they are marked "Global". They also have a central region bucket (data center) located in Council Bluffs, Iowa. If I relocate my stuff to the heartland, things might be a little faster for most people. Without a doubt, things would be much faster if I was using a real database like Mysql on the backend. That is a good longterm goal for sure. As for the application code, I would rather not rewrite it from scratch. Changing the data access calls to talk to Mysql instead of Google's sheet api would not be that difficult but also not something I can do overnight. My strategy going for now is to continue to monitor for a week or two. If I see submit times consistently over one minute, then I will start paying Google for a couple of things which give me a little more control like where my stuff is running from. Google service charges are all based on usage and I dont see this app as being that much of a resource hog. Besides, I think there would be considerably less data traffic out in the midwest. Everyone is etiher picking corn or going to football games. ;)
Right, I didn't really mean doing anything with it during the season, but more of maybe a project to work on after the season wraps up (I assume it's an entirely different application from the basketball one, right?...are there any "shared" pieces between the two?). We can start talking about ways to make it a bit more efficient and stuff. I don't really know too much about google's programming stuff and backend database structures (although I do use the consumer-grade Google drive quite a bit). If you want to see about setting it up using php and mysql at some point once things are all wrapped up, I could definitely help you with that. I've written a few php/mysql apps that involved multiple users logging in to access data...and I've learned a TON about query optimization and reducing server load by caching data since then, so we could probably figure out something that works well.

I don't know how much work goes into things behind the scenes on a "weekly" basis (do you do a lot of updating of things manually, or do you have code that scrapes sites to gather the data like scores and things?), but if you have time where you're not busy otherwise and you're not busy managing the pick'em app, we could start putting together a demo/test application.

Just an idea, if you're interested.
Scott Lawson - Board Admin
Western Illinois University Alum/Fan/Employee
Member of the Marching Leathernecks - 1996-2000
User avatar
sealhall74
Posts: 5770
Joined: Fri Apr 18, 2014 1:18 pm
Location: Wherever, Windblows

ST_Lawson wrote: Tue Aug 29, 2017 8:34 pm
sealhall74 wrote: Tue Aug 29, 2017 6:42 am
My backend is nothing more than a spreadsheet on my Google Drive and an App Script (javascript) project also out on my Google Drive which generates the html for the user interface along with inserting/updating rows in the spreadsheet based on user game picks. I am not using any of the pay services offered by Google yet. I assume that my docs and apps are located on Google's Northern Virginia regional bucket because that is close to where I live but I dont really know for sure, all I know is that they are marked "Global". They also have a central region bucket (data center) located in Council Bluffs, Iowa. If I relocate my stuff to the heartland, things might be a little faster for most people. Without a doubt, things would be much faster if I was using a real database like Mysql on the backend. That is a good longterm goal for sure. As for the application code, I would rather not rewrite it from scratch. Changing the data access calls to talk to Mysql instead of Google's sheet api would not be that difficult but also not something I can do overnight. My strategy going for now is to continue to monitor for a week or two. If I see submit times consistently over one minute, then I will start paying Google for a couple of things which give me a little more control like where my stuff is running from. Google service charges are all based on usage and I dont see this app as being that much of a resource hog. Besides, I think there would be considerably less data traffic out in the midwest. Everyone is etiher picking corn or going to football games. ;)
Right, I didn't really mean doing anything with it during the season, but more of maybe a project to work on after the season wraps up (I assume it's an entirely different application from the basketball one, right?...are there any "shared" pieces between the two?). We can start talking about ways to make it a bit more efficient and stuff. I don't really know too much about google's programming stuff and backend database structures (although I do use the consumer-grade Google drive quite a bit). If you want to see about setting it up using php and mysql at some point once things are all wrapped up, I could definitely help you with that. I've written a few php/mysql apps that involved multiple users logging in to access data...and I've learned a TON about query optimization and reducing server load by caching data since then, so we could probably figure out something that works well.

I don't know how much work goes into things behind the scenes on a "weekly" basis (do you do a lot of updating of things manually, or do you have code that scrapes sites to gather the data like scores and things?), but if you have time where you're not busy otherwise and you're not busy managing the pick'em app, we could start putting together a demo/test application.

Just an idea, if you're interested.
When you say consumer grade Google drive, what does that mean? You are paying for additional space?
Embrace the pace of the race.
User avatar
ST_Lawson
Site Admin
Posts: 8247
Joined: Wed Apr 16, 2014 1:15 pm
Location: Macomb, IL
Contact:

sealhall74 wrote: Thu Aug 31, 2017 10:30 am When you say consumer grade Google drive, what does that mean? You are paying for additional space?
I just meant the free Google Drive for regular consumers: https://en.wikipedia.org/wiki/Google_Drive
I am not currently paying for anything extra.

I do get access to G Suite for Education through Western: https://en.wikipedia.org/wiki/G_Suite
So if you needed more space or something, we could use that, or the server that I this site runs on (or that the mysql database for it runs on).

If we ever needed something a bit more substantial, we could look into setting up something on Amazon Web Services. They have a free tier/trial (https://aws.amazon.com/free/) that would probably cover everything that would be needed for a year, otherwise running a basic mysql instance through them is probably not horribly expensive.

Although really, probably just converting to mysql and php on the server I have would improve the speed and accessibility significantly.
Scott Lawson - Board Admin
Western Illinois University Alum/Fan/Employee
Member of the Marching Leathernecks - 1996-2000
User avatar
sealhall74
Posts: 5770
Joined: Fri Apr 18, 2014 1:18 pm
Location: Wherever, Windblows

One improvement I can easily do is to short-circuit the submit process. One step in the process is simply to get the user's picks into a spreadsheet staging area. I just did a test and that step happens very quickly, like in 2 seconds. The second step is to move the user's picks from the staging area into their final resting place, complete with formulas intact and all of the rows sorted properly. That step is very costly.
I could stop the user submit process after the first step and return control at that point. The downside is that the picks would be in a "holding pattern" awaiting final post. The posting step could be done every hour or so by a triggered process. Going this route would mean that the user would probably want to login again at some later time to verify the posted picks are in place, or I could send an email with the currently posted picks after the triggered update process, or they could just trust the system to work. This is not perfect but may be better than what I have now. The only real issue is a user interface one, i.e., how to differentiate the posted picks from the staged picks on the picks form page.

FYI, the same code base is used for both roundball and pigskin, and even multiple pigskin contests which is what I did one year. However, each contest is managed from its own spreadsheet. My handholding during the week is minimal. I run something to scrape the schedule off of the NCAA scores site. During that process, I also scrape the Massey site to get the game matchup data. I also scrape the final scores from the NCAA site on gamedays and that process can also run via trigger how often I want it to. The boxscore for the Statpick game is scraped from the Conference site or sometimes I just enter it manually. Once the game scores are entered, the formulas kick in and it all works without too any headaches. Honestly, the most time-consuming part is sending out the e-Gift awards at the end of the week. I dont think I can automate that but a browser macro recorder might make it easier. Amazon seems to be the site that is most popular and that one is fairly quick to do.
Embrace the pace of the race.
Post Reply