The 12th ICFP Programming Contest FAQ

• Q: Are you going to use the most recent submissions, or the most recent working submissions.
• A: The most recent working submissions, before 13:00:16

• Q: Will the scoreboard be removed? Will the auto tester be available till 13:00?
• The scoreboard and leaderboard will be be removed with 4 hours to go. The auto testing will be disabled a few minutes before 13:00, to allow the server to focus on accepting your last minute entries and source uploads. Anything uploaded before 13:00:16 will be accepted, but do not count on your teampage being updated in the last hour.

• Q: Will satellites always rotate the same direction as you in your test cases?
• A: Yes, except perhaps satellites round the moon.

• Q: We're having problems with submission of attached trace for scenarios in problem #3. For instance, attached trace for 3001 works ok on our simulator and produces a score. In fact, we get within 55m of the target and stay there for >900 seconds without firing the engine. However, when I submit this trace, I get timed out error instead of a score. Our simulator appears to work correctly, since we have been able to work through other scenarios (except for problem #3). Can you explain why we're seeing this result?
• A: It is good practice to check where you are is where you think you should be, and compensate for any simulation or rounding issues.
  (This example failed for us on 3 different VMs, on different architectures, and appears to settle into an orbit 4000M away. We realize this is frustrating, we recommend adding compensation code, because it can read live data from our VM-based sensors.)

• Q: My program keeps crashing! Why? It works fine for me.
• A: Here is a trace of the example running:

  ./hohmann replayXXXX.osf 
  0xcafebabe 693 1004
  stepto: 0
  0 2 :  < 2 -2427.97 >  < 3 2.96125 > 
  stepto: 1
  CRASH on 1
  

  Table 6 in the document has a configuration port (0x3E80). This is an inital change the input memory, and must be included in deltas in the output file. So typically the first entry in any submission file format will be the writing of port 0x3E80.

WeightedScore = (SUM of Scores for all 1001 to 3004) +
                f(Score for  4001) + 
                f(Score for  4002) + 
                f(Score for  4003) + 
                f(Score for  4004)

f (x) = if x > 200
        then ((x - 200) * 4) / 3
        else 0

• Q: In problems 400x, the specs mention a refuelling station, plus satellites target0 to target11. However, there seem to be only 11 satellites. Is this an error in the specification?
• There are 12 satillites total, including the refueling station. We have corrected this on the document, v1.8.

• Q: What machine are you running your VM on?
• Redhat, Fedora release 10 (Cambridge), x86_64, gcc -O2 -lm

• Q: Our solution model currently produces very large solutions (>1MB). These are valid solutions, as far as we know, but they are rejected by the uploading system with a database error.
• A: Because of resource limitions, we need to have some sort of limit. We will accept .bz2 versions later this afternoon, if this helps. 1M represents > 22,000 burn event ajustments, which should be plenty.
  
  Resource clarification: We do not accept solutions > 1M, after compression.

• Q: Its possible you have some issue with validation of task solutions. If you check our submissions for scenario 3003 [...] They all marked as "timed out" although our simulator prints valid score at time step around 0x8099. Then we found a different solution with score time equal to 0xB69, and it was accepted.
• A: Make sure you are actually close to the target. Staying at approximately 999 meters may cause rounding differences. It is good practice to check you distance, and compensate,

• Q: As a quick point of clarification, does firing the thrusters during the 900 seconds cause the timer to restart? We have a run of Meet and Greet that appears to stay within a kilometer for well over 2000 timesteps but that fires thrusters frequently but the score remains zero. Should a solution that fires thrusters frequently but that stays within the target distance pass?
• You need to achive a stable orbit close to the target, without firing thrusters for 900s.

• Q: How is the leaderboard score determined?
• The intention was the summation of the scores for each scenario, where the first group of simulations were worth a total of 400, the second 800, the next 1600, and the final group, 3200.
  One team (who will remain nameless) pointed out that doing nothing on any of the problem 4001 to 4004 gives a score of 200 for all of the four problems. (No fuel used, no satellites met). Rather than have everyone run these, we are weighting the leader board.
  The formula is

  WeightedScore = (SUM of Scores for all 1001 to 3004) +
                  f(Score for  4001) + 
                  f(Score for  4002) + 
                  f(Score for  4003) + 
                  f(Score for  4004)
  
  f (x) = if x > 0 
          then ((x - 200) * 4) / 3
          else 0
  

  This restores the original intent, without changing the individual scores of each test. We do not believe it will make a difference to final 10, and makes the leader score board a more accurate metric of performance.
  So the output score for a specific test remains the same; this only effects the Weighted Scores for the leader board, specifically to avoid everyone needing to run pointless simulations, and an 800 point score inflation.

• Q: Why did the final binary get patched?
• A: The first version of the final binary, MD5 b7c388efcb7cf7e26e430a75394be8ac, has some scoring infidelities. Specifically, the scoring algorithm was inverted, and the refulling actions could leak fuel, and other smaller issues.
  
  So we have updated the binary for Operation Clear Skies. The third and final 400x binary has MD5 db6906572307234f36f2830f8b618c43.
  
  
  
  The problem(s) to be solved remains identical, and previously submitted solutions for 4001 to 4004 have been retested and rescored automatically.

• Q: Any other prizes, given we now know the contest?
• A: Yes. We will be giving a (cashless!) prize for the best visualization, and the best misuse of the VM.

• Q: If I will not submit my source code before lightning round that means that I will not be able to compete further?
• A: You do not need to submit anything for the lightning round! Someone could start on Sunday, and still take part in the main contest.

• Q: Is it possible/legal to clone the VM?
• Q: Will it be possible for our solution to read the input binary multiple times? If possible, it would allow us to run the scenario multiple times within a single run of our solution.
• A: As long as there is a primary VM, cloning it should be fine. Does the solution run only taking into account the output of the primary VM? Running a secondary one to try things out as part of the response to the output is fine (just computation, not cheating or looking inside the primary VM).

• When computing the score for our trajectory for scenario 1001, we found that our computed score disagreed with the score that the virtual machine computed. At the end of the run, we had 6050 (of 10000) fuel remaining, and elapsed ~19778 time steps. Using the formula in the problem specification, this is a score of 77.92. However, the virtual machine outputs a score of 67.7724. We could obtain this value by using the "fuel used" instead of the "fuel remaining" in the score formula -- that is, the virtual machine was basing our score on the fuel we had consumed instead of fuel remaining. Also, we think the problem specification could be clarified by saying that the score formula uses the integer logarithm of the time remaining divided by 1000, rather than the exact logarithm.
• A: The given binaries are the specifications, and the formulas may be approximations. Use the binaries!
  The formula for the time component is 30 points for the first 1000 seconds, then 29 for 2000 seconds, then 28 for 4000 seconds, then 27 for 8000, etc.

• Q: The fuel scoring in the first binary appears backwards. Is it?
• Q: It seems the scoring scheme is wrong. Our program gets better score when we accomplish the task with LESSER fuel remained.
• Q: ...
• A: Yes. It is. Burn as much fuel as possible on the first example, but not too much! Doing two transfers is one possible solution here.
  The published fuel formula is the correct direction for problem 2 and 3.

• Q: The task description states that the top 10 teams will go through a validation process on hidden scenarios. Does the same apply to the lightning division?
• Q: The task description states that the top 10 teams will go through a validation process on hidden scenarios. Does the same apply to the lightning division?
• A: The same principle, but the number we pick to execute is at our discretion, given it is just a single prize.

• Q: I have discovered that although the score for a successful run is placed in the slot of the trial that is indicated in the header, the trial that is actually run is based on what config number is sent to the config port in frame 0. This means that by marking the header with the scenario 1001 and sending data for scenario 1002, the run will be accepted but the score will be placed under scenario 1001. This could be used to submit scores that are higher than would otherwise be possible for a given scenario (and possibly used across different problems, although I have not explored this flaw, only accidentally encountered it).
• A: All such submissions will be marked as BAD FORMATING at some random time; please do not do this. The number in the header should be the same as the config sent to the port number used. We use the header to decide which VM binary to run against, but will invalidate all such bad solutions.

• Q. Does status = CRASHED! on the submission page mean that my satellite has crashed into the Earth, or does it mean I submitted a bad file? thanks!
• A: CRASHED! means you hit the earth, or you tried to use too much fuel, or the submitted binary somehow returned -1.

• Q. What should I do to participate in ICFP Programming Contest? Should I register somewhere or anything else?
• A: Pre-registration is not required. There will be small downloads required, including the problem description. The registration will ask for
  • Name of team
  • Contact email address for team
  • (optional) URL of blog or webpage for team

• Q. Why June? Many students have exams till the very end of June. It would be nice to postpone contest for 5(or more) days.
• A: We are sorry the dates do not work for you, but every time of year has commitments for someone. The dates are picked to allow evaluation of contest solutions before ICFP. Since 2000, the contest has been held June twice, July four times, August twice, and once in both June and July.

• Q. Will there be a teaser this year too that tells something about the contest task in a cryptic way but you can't really infer anything from it until the task is out, such as the ants image in 2004 or the codex in 2006?
• A: There already are a number of teasers.

• Q. Will you officially tell on the contest webpage who the organizers are, or will we have to infer from conference webpage?
• A: Sure. We'll add a list of names shortly.

• Q. Will we need a continuous internet connection to run something on our machine which communicates remotely with your servers? (There's no real precedent for a submission like this in previous contests, only approximations like the on-line game servers in 2002, but internet connection is getting more and more available each year so maybe it's time for this finally.)
• A: No.

• Q. Will there be a pre-registration form or a large encrypted download or anything else for which we'll have to watch the contest website a few days before the contest?
• A. Pre-registration is not required. There will be small downloads required, including the problem description.

• Q. What prizes are there (eg. first prize, second prize, judges' prize, lightning first prize, third prize etc) and what can the teams win for them (eg. the traditional proclaiming of "the programming language of choice for discriminating hackers")?
• A: First place, lightning round, and judges' choice prizes will be awarded, which will include prize money to help defray the costs of travel to the conference for the winners as well as small cash prizes. There will be other cashless prizes announced (second place, etc.) at ICFP.

• Q. Will there be an official wiki where we can discuss the contest after it's over, such as there was in 2008, or at least a webpage that links to the webpages of all participant teams, such as in 2000?
• A: No, but we will encourage teams to provide a website with submissions, so we can document team participant on the 2009 contest archive page.

• Q. Will there be official mailing lists, wikis, irc channels, etc where the contestants can communicate with each other and possibly with you when the intent is not an official question to the organizers? It would be important to know this because if you're planning none of some particular medium (eg. irc channel), we shall start an unofficial one as early as possible so it's more likely there's only one of it.
• A: There is no official mailing list, wiki, irc channel, etc. During the contest, we will monitor
  • the contest email address icfp-contest@ittc.ku.edu, and
  • the IRC channel #icfp-contest on irc.freenode.net.

• Q: Will there be official contest mailing lists where we can contact you with public questions and you send official announcements?
• A: All information will be disseminated using web technologies, including the contest RSS feed.

• Q As the other direction of the above, will you be releasing executables specific for the contest that we may want to run on our computers, like the simulators for the 2008 and 2005 contests? If so, could you tell in advance what systems you are planning these will be working on?
• A: N/A

• Q: You write in one FAQ that we will have to send programs you run on your computers, and that you will try to provide multiple architectures so more people can participate. Could you perhaps tell us in advance what particular systems you are planning to have, so those who like those platforms would be happy and calm, and those who don't could nudge you in email to include their favourite systems. (I myself would prefer linux-amd64.)
• A: We are not providing specific support for any specific system. Feel free to compete with any system.

• Q: What will be the maximum allowed team size this year?
• A: Any size team is permitted.

• Q: On the contest home page, it says the contest will start a 13:00:16 CDT and run for 72 hours, but the contest blog says it will end at 13:00:16 EDT on Monday which would only be 71 hours. Which is correct?
• A: The contest lasts for 72 hours, starting 13:00:16 CDT, and the blog entry has been corrected.

• Q: Will we have to produce an executable that runs on a particular environment like last year or will we only need to be able to run our program on an environment of our choosing like the years where the only thing submitted was data produced by the program?
• A: The organizers want give as many OS platforms and languages as possible the opportunity to compete in this programming contest.