|
Post by jimmy on Apr 23, 2013 0:43:42 GMT -5
Well, the first month of races is coming to a close. We're still waiting on Codge and Japo to finish up the last leg, and another registered racer (ANTnDEC) may still get in some flights. I've finished tallying up all of the scores (so far), and the current front runner is Kronzky by a large margin with a 12-minute race time. Rounding out the back of the pack are myself and PRB with times up in the 40s. You can view the spreadsheet I used for the math here: docs.google.com/spreadsheet/ccc?key=0AmGGITW0iKSNdGNPVEtpQzg5cmRiaHc0RzNtU0RjUXc&usp=sharing&gid=4A quick interpretation guide: There are 3 (or 4?) columns for each leg... the white column (RCT) is the "Reference Cruise Time", which is auto-calculated based on your declared aircraft's cruise speed divided by the leg distance. This is the time that you are "competing" against, and is (in theory) what your fastest possible time could be in ideal conditions. The yellow column (AFT) is the "Actual Flight Time"... this is your flight time as indicated by the Duenna. The green column (Leg#) is your "calculated flight time"... this is the actual time that is worth points, and is your AFT-RCT. These green columns are totaled up for the "Total Race Time" at the end of the row. Keep in mind that many times can still change... my crash on leg 1, and others' crashes on leg 4 are currently calculated to the "slowest successful time plus 1 minute", but could change if anyone else flys those legs at a slower time. First of all, a big thanks to everyone for their inputs as we try to get this thing off the ground. It was definitely a lot more of a project than I anticipated it to be. Also, another big thanks to Teson... it was his brainchild that came up with the handicapping system that we all voted for. Plus he helped me build that scoring spreadsheet, and has been helpful in getting the aircraft database populated so that we could all declare that aircraft we were going to race with. Now that we've all gone 3 or 4 rounds with the Duenna, and most of us are still standing... what kinks are there to work out? What improvements might there be to make things easier or more fluent?
|
|
|
Post by kronzky on Apr 23, 2013 8:36:13 GMT -5
Did you link to the right spreadsheet? This is the one with the aircraft data.But anyway, thanks so much for setting this up, Jimmy!It's a great concept, and must have taken a lot of work. I enjoyed every part of it, and hope there'll be lots more! I hope it's all automated? Or otherwise you'll be swamped, once this becomes really popular! (Which I'm sure it will sooner or later.) So here's my thoughts on how we might make it even better: First of all, I'm afraid the handicapping system probably didn't work too well. I voted for it too at first, and was very impressed with what teson1 had come up with (it was obvious that some very creative thinking went into the whole thing), but unfortunately it fails to account for the different planes' climbing rates. The speed differences are covered well by the system, but if a plane (due to its superior power) can take a shortcut over some steep cliff, whereas the less powerful planes will have to take the long route around, then this time/distance savings is not taken into account. I did some benchmark tests before I actually picked a plane, and it was very obvious from the second leg on that raw power will still win, as I could fly right up the steep ridge next to the airport (purple path), when other planes had to take detours: So we might have to rethink the whole classification thing once more... I can't think of a better system myself, and I'm afraid there probably won't be one. (I used to be involved in car racing, and they've done handicapping systems for decades, and there will still always be the same cars that win their classes.) The second thing we might have to think about is tasks like "follow the river" or "follow the road". Since this is a race people will cut corners (quite literally). In car racing this comes with a natural penalty (bumpy, bad traction, etc), so competitors will keep it to a minimum. But in air racing no such automatic penalty occurs, so I don't know how we can enforce that the corners that are cut aren't too massive. We may just have to restrict these "follow" tasks to tight canyons, where you don't have another option than to stay on track, or you will run into a wall. But other than that, I think you got a pretty good thing running already!
|
|
|
Post by jimmy on Apr 23, 2013 11:16:38 GMT -5
Did you link to the right spreadsheet? This is the one with the aircraft data. thanks for the feedback. Yes, that is the right spreadsheet... There are seveal tabs, one of which is the April Leaderboard. I've adjusted the hyper link to open to the correct tab. that's my biggest concern at the moment, too... Nothing is automated. And while the Duenna keeps good records, digging through them to find all of the relevant info was kind of a pain in the ass. The first thing I noticed was that the individual flight pages have some incorrect data at the top summary area. "maximum altitude" for example... Kronzky's was listed at 900-something feet, which had me confused since the Trout Lake field had an elevation of 13xx. The duenna jpg had the correct info, so that was "easy" to overcome... But right now, the spreadsheet has everyone's "incorrect" max altitude listed, which I need to re-accomplish to figure out the bonus minute.
|
|
avian
Student Pilot
Posts: 36
|
Post by avian on Apr 23, 2013 15:13:45 GMT -5
Trout lake airstrip on my machine was at around 1900ft rather than 1300, some anomaly there?
|
|
|
Post by kronzky on Apr 23, 2013 16:45:44 GMT -5
Yeah, Duenna seems to be a bit confused about the altitudes. I measured 1,923' in FS9, and 1,918' in FSX. (And according to Google Earth it's supposed to be around 1,910'.)
Perhaps some people have alternative meshes? But then the whole "who's lowest" wouldn't make sense anymore.
|
|
teson1
Commercial Pilot
Posts: 243
|
Post by teson1 on Apr 23, 2013 18:30:33 GMT -5
> kinks, improvements
Frankly, it's a great set-up already - very viable and interesting. Nothing fundamental to change.
Keep the circuits as interesting and varied as the first ones and it should be enough to please both the competitive pilots as well as the leisurely crowd just along for flying through interesting scenery, and keep all coming back for more.
I agree with kronzky that the handycap system is not perfect.
It is encouraging that the Scout is at least able to mix in at the top even with the simple current system (though it has to be recognised that avian flying the C185 (BS) would probably have been right up there with kronzky if he hadn't had that aborted flight on leg 4 that counted as a crash - very tight flying both, likely better than mine - congrats).
However, the slow planes seem to be clearly disadvantaged on the legs where significant climbing is required indeed. Not only because more powerful planes can take a more direct path if a very steep climb is required (reduced flight distance), but also (mainly) due to the fact that the climb time is longer (longer at slow speed =time lost vs cruise speed).
Very preliminary analysis of flights 2, 4 and 5, comparing teson1 and kronsky's flights indicate that climbing through 7000 ft adds about 40-80 seconds for the Scout (ref speed 120 Kts) vs the C185 (155 kts) to total flight time.
Further testing under well controlled conditions ("clear weather", no wind) with a wider range of planes, and the same pilot flying for consistency of results, would be needed to more clearly determine the impact of climb performance of different planes on the flight time. I'm interested to conduct such further flight tests.
Once we understand more about the performance of different planes during climb it might be possible to tweak the MCT attributed to each leg to better account for the different climb performance of the planes.
Ideally this correction factor could directly be calculated from the required climb of the leg (MFD and the vertical climb height of a leg would enter into calculation of MCT).
If that works, a disadvantage would still be that MCT (should rather be called "Reference flight time" RFT) would be less intuitive to grasp, become a formula. So the question is whether pilots would accept to compete against a "formula".
But before we could accept such a method we would require solid data that demonstrates that the playing field is indeed more level if applied.
Unless there is consensus that this is not something we want to pursue, I would be willing to run such tests (I enjoy understanding the performance of planes and working on this type of projects so no worry there). Once we have data we can decide whether this is something to go further with.
But I won't get this done before the May circuit, so I'd suggest we go on with the system as is for now, unless somebody has a clever idea on a new system.
Gunter
|
|
teson1
Commercial Pilot
Posts: 243
|
Post by teson1 on Apr 23, 2013 19:03:47 GMT -5
Another note regarding improvement. I suggest that MFD is determined accurately (to about 1 nm) (or even overestimate to advantage the slower planes) for those legs where a direct flight path is not possible. (I had said to Jimmy that a rough approximation is likely enough, but don't think so any more). Example: Leg 2 around Mt Hood. MFD=38.7nm. Real minimal flight distance going around the mountain at 9000 ft is about 42 nm I guess. the additional about 3-4 nm take estimated about 1.5 minutes for the C185, but 2 minutes for the Scout (and 3-4 min for the J-3 Cub), further disadvantaging the slow planes. Measuring the correct flight distance by following the contours is possible in skyvector by placing a course along the altitude lines (these are 500 ft apart) or with flood in Google Earth. PS: My plane has shown 1919 ft parked at trout lake so I guess there are no wide variations in airport altitude. No problem there I believe. I have enjoyed the challenge to approach in the river bed and sneak in to the airfield between the trees a few feet off the ground. PPS: The correct Max altitude (and flight/baton time and point2point distance between take-off and landing location) has to be read off the duenna .jpg graph (or the .txt file). Duenna uses several levels of resolution. The distance/altitude/track information shown on the flight's Duenna site is not reliable as you have remarked. The second tsv file also has high (about 3 seconds) resolution.
|
|
|
Post by jimmy on Apr 23, 2013 22:41:00 GMT -5
It's interesting that you mention running some test against data for high-climb rates... the first three courses were all designed many months ago, and the next course will have an immediate climb of almost 6000 feet (maybe higher, but in that area) for 2 of the 3 legs. But unlike this first course, it's all designed to fly in a straight line. There will probably be a big disadvantage to the Scouts (et al) because the distance between take off and the first mountain peak that must be directly overflown is quite small - without going back and looking, I assume that most planes will probably have to make a circling climb, but I suspect that the 185 could probably get to altitude before reaching the first obstacle at the maximum rate of climb configuration. At least one of us (me) will continue to fly a slower plane - at least we can get some data after the fact, if nothing else.
The third course will probably be more fair for the smaller planes, because the whole course will be flown at low altitudes. There will also be no direct routing, and no prescribed "following". You must simply get from point A to point B in a mountainous region, following whatever route you choose without going higher than a given altitude, knowing that a direct route will result in a "controlled flight into terrain".
As for automation... I would love to find someone up on the technology to automate as much as possible. I know that there are several flight verification clients out there for VAs and such, with PHP backends to track things. If anyone could develop a system to "plug in" the race course, which whatever client uploads the data to, and the scores are popped out automatically would be awesome... although probably a far fetched idea.
|
|
|
Post by kronzky on Apr 23, 2013 23:56:07 GMT -5
Further testing under well controlled conditions ("clear weather", no wind) with a wider range of planes, and the same pilot flying for consistency of results, would be needed to more clearly determine the impact of climb performance of different planes on the flight time. I'm interested to conduct such further flight tests. I already did a few tests like that myself before I picked my plane, and in the process developed a little gauge that will give me exact timestamps at the flightplan's waypoints, as well as at the touchdown or the time a specified altitude has been reached. (For testing I only do the takeoff myself, and, for consistency, let the autopilot do the rest.) I found it quite useful to benchmark different planes, and also to compare different routes and approaches for the race course. In the screenshot below I took off after 10.4 seconds, passed 4 waypoints, and reached an altitude of 8,000ft after 473.9 seconds.In case you don't have something like that already, it might also help you do further tests for the handicapping system. You can download it from here: www.kronzky.info/fs/wptimer/wptimer.zipI also noticed that the distance calculations in Duenna were off at times, so I wrote a script to do the calculations myself, and it seems to work a bit more reliably. It can also deal with those KML files that are somewhat corrupted, i.e. where you constantly have lines going from your current position back to some previous one. It will ignore those corrupt segments during the calculations, and create a new KML file with those parts omitted (so the file can be properly displayed in Google Earth, for example). The link for that script is here: www.kronzky.info/fs/fixkml.zipAs for automation... I would love to find someone up on the technology to automate as much as possible. I know that there are several flight verification clients out there for VAs and such, with PHP backends to track things. If anyone could develop a system to "plug in" the race course, which whatever client uploads the data to, and the scores are popped out automatically would be awesome... although probably a far fetched idea. The VA I fly for is using a system like that (provided by simpilotgroup.com), but it's really bare-bones, and the data it gives you is far less detailed (and just as buggy) as Duenna's. It only shows departure and destination airport (without a detailed flightpath), and the flight statistics contain very little detailed information. There may be other systems out there, but I wouldn't consider this particular one an improvement over what we have right now. P.S. Some more input on a handicapping system. The more I think about it, the more I'm afraid that it just won't be possible to create a fair and even one, that levels out the different performances. There are just too many variables, all of which affect your time, and which will be impossible to be caught in one formula. Just some examples: - Since only max speed is considered, planes that reach it faster than others will have an advantage (which is normally slower planes).
- Some planes (e.g. 185 & Aviat) can do a full-throttle start (stay on the brakes until you have full RPMs, then release), which shortens the runway needed.
- There are differences in how fast flaps are lowered (e.g. 185 vs. Aviat). This will affect the slowdown time needed before landing.
- Some planes can do significant overspeeds without taking damage, while others blow up as soon as you barely exceed cruise speed.
- Slower planes are easier to handle, so you're less likely to make a fatal mistake.
- And so on, and so on...
Even if the difference is only seconds — everything else being equal, a second is all you need to win a race. I'm afraid the only way to level the playing field is by only using one plane. It could be a different one for each race (and should, of course, be free), but that way it would really only come down to pilot's skills, and nothing else.
|
|
teson1
Commercial Pilot
Posts: 243
|
Post by teson1 on Apr 24, 2013 16:03:00 GMT -5
kronzky, went to your web site. www.kronzky.info/fsWow ! Very nice gauges ! Good work !! I downloaded most of them (including the sextant one which I had planned to get anyway). You are right that we can never hope to design a handycap system that completely equalises performance of all planes. What only we can hope for is to have a system that brings planes to a potential to score within, say, 1-2 minute per flight in the most unfavorable case (say, the cub), and most plane less than that. But we're not nearly there yet, and not sure whether it is even posible ... That gives at least a small chance for a skilled and lucky pilot flying a disfavored plane to score close to the top (but the other pilots have to make errors for him to achieve that). The whole objective of such a handycap system is to allow pilots to fly a plane they like and care for, and still have a fighting chance. It clearly does have drawbacks. We've realised that there still is a significant difference in scoring of different planes. And I think the pilots flying the Pacer have also realised that it can't compete as the cruise speed is too close to the barber pole, so that pilots can't really fly at the maximum speed. That system clearly would benefit from some improvement. The question is do pilots prefer to fly a plane of their choice, and be potentially disfavored, or do they prefer everybody to compete on an equal footing) ? Your proposal indeed is one that would place all pilots on equal footing (we'd still have to find planes in FSX and FS9 with same performance, or restrict to FSX). The question is whether pilots would be interested in such a system. They may have to fly planes which they're not interrested in. We'd have to look what quality planes we find. The Mikko Maliniemi Maule comes to mind as an excellent quality plane (that I'd be delighted to fly). It's definitely a valid proposal if we want to have all pilots on a completely equal footing (and I may have misguided people in thinking the performance would be completely leveled out by the handycap system ...). Why not try out if and see how we like it ? Now that we have the BPRL going and a group of pilots interrested, maybe Jimmy could set up one of the circuits or even legs with a fixed plane for all to fly so we could try out how we like it ?
|
|
|
Post by kronzky on Apr 25, 2013 9:52:36 GMT -5
Very nice gauges ! Good work !! I downloaded most of them (including the sextant one which I had planned to get anyway). Yeah, this little race totally got me distracted from my sextant project! It's pretty much done, but I still gotta finish the documentation. PLUS — I finally found out how to calculate the *actual* sun position in real-time (not the smoke and mirror position that's used currently), so I'm really struggling to decide whether to invest the work to implement that, considering that it might only be used by a handful of people... We might want to continue this discussion in the public area of the forum (e.g. in the Thoughts on Aircraft "Fairness" thread), as it concerns even those (or especially those) who haven't signed up yet. As far as the single-plane option is concerned — we could perhaps even set up a poll where we offer different planes, and then people can vote on which should be used for the race.
|
|
teson1
Commercial Pilot
Posts: 243
|
Post by teson1 on Apr 25, 2013 19:04:38 GMT -5
Very nice gauges ! Good work !! I downloaded most of them (including the sextant one which I had planned to get anyway). Yeah, this little race totally got me distracted from my sextant project! It's pretty much done, but I still gotta finish the documentation. PLUS — I finally found out how to calculate the *actual* sun position in real-time (not the smoke and mirror position that's used currently), so I'm really struggling to decide whether to invest the work to implement that, considering that it might only be used by a handful of people... Wow, that sounds great! Even closer to how a sextant is used in reality ! And it would allow to take a star shot, and only then look up the Hc and Zn in the tables, like in reality. And you'd have to be really careful to look up the right values, and also pay attention to timing the shots to the second ... Nice ! Looks like quite a project though. And you probably wouldn't have a very large public indeed. I'd use it !
|
|
teson1
Commercial Pilot
Posts: 243
|
Post by teson1 on Apr 26, 2013 1:31:06 GMT -5
Why not try out if and see how we like it ? Another suggestion which combines about all options (handycap/defined plane/classes) would be to define a freeware plane as reference plane for a race, which would be the fastest one, but pilots could select any slower one if they prefer, accepting being slightly disfavored.
|
|
avian
Student Pilot
Posts: 36
|
Post by avian on Apr 27, 2013 14:50:12 GMT -5
Wonderful gauges on your page Kronzky!!
|
|