apo-wiki

Attending: Nancy Chanover (NMSU), Kevin Schlaufman (JHU), Zach Berta-Thompson (CU), Aleksandr Mosenkov (BYU), Bill Ketzeback (APO), Eric Bellm (UW), Misty Bentz (GSU), Russet McMillan (APO), Anne Verbiscer (UVa), Moire Prescott (NMSU), Joanne Hughes (Seattle U), Michael Hayden (OU)

• Seattle - nothing to report
• Washington - nothing to report
• Wyoming - xxx
• BYU - nothing to report
• Colorado - nothing to report
• JHU - nothing to report
• NMSU - confusion about password mailing
• Oklahoma - nothing to report
• Georgia State - nothing to report
• UVa - nothing to report
• NAPG - no report

The detailed site report is included below, followed by additional information discussed during today's meeting.

3.5-m Telescope and Instruments Highlights, 1/08/25 – 02/04/25

1) Overview

Weather wise, we had a very cold January with a mix of clear and cloudy nights. Numerous cold related problems were present in January, detailed in the following report. There were no astronomers, instrument teams or classes that visited APO in the month of January.

2) Operations

The main site generator has been experiencing a few problems since the start of the new year. Fuel contamination is suspected and we obtained some test kits to determine the cause. We are waiting for the results to see if it is bacterial or water in the diesel. A repair of some wiring was also required as the main electrical transfer switch was not sensing enough voltage from the generator to switch over on a drop of utility power. In the meantime the backup generator has been on standby.

An issue with the computer UPS occurred after a planned replacement of three quarters of the backup batteries. Batteries were only charging to a fraction of their full load and runtime was quite short. The problem has been resolved and the unit is now 100% charged with the expected hour of backup runtime should we lose all utility and backup power. This would be enough time to gracefully shutdown servers.

3.5m Telescope: Telescope is working as expected. Following a planned power outage for line maintenance by the local power company, there was a failure of a circuit breaker to the enclosure motor amplifier cabinet that prevented moving the building one night. Disassembly and cleaning the breaker the next day allowed us operate until a replacement could be obtained and installed.

0.5m Telescope: Telescope is working as expected.

KOSMOS: Experienced multiple failures of the slit viewer shutter when temperatures dipped near and below the -10C rating. We replaced the shutter with a spare. Some adjustment to guide optics was also needed as the image scale was changed during the shutter repair. Additional observing overhead was required in the meantime to center up targets using the science detector in imaging mode. The disperser and mask wheels also have been struggling with colder ambient temperatures during this time, making use of the instrument challenging during part of the colder nights.

ARCTIC: Maintenance on the diffuser mechanism will be scheduled as well given the reports of intermittent behavior at times.

Agile: The camera is back from the repair house for local bench testing with APO software for failure to software select gain settings as well as a demo camera for testing. The earlier problems with the TEC and readout of the CCD have been resolved. We are waiting to hear back from the repair house if they learned enough from the testing we did to try again to repair the camera or if we will reinstall with manual gain settings.

ARCES: One reported failure of the sky/cal mirror was reported. This certainly is another cold temperature related problem as the motor driver is at the limit of its current output. The system is currently usable.

DIS: System is currently up and usable.

NICFPS: System is cooled and usable.

TripleSpec: System is up and usable. There was one report of the main detector reading all zeros. A reset of the camera controller corrected the problem.

APOLLO: Installed a new chiller for the laser. We have ranged a few times since installation without seeing abnormal operating temperatures. The system is in operation.

Site had few power related issues recently. Fuel contamination in backup generator. Wiring issue in generator also corrected. Full service of main generator is complete and it is ready to be put into service as full back up utility power (as of this morning). Computer room UPS - servicing in January. Weren't fully charging but investigating get that unit back on service contract so if components fail it's not as expensive for us to repair.

During routine scheduled power outage after coming back to main utility power, breaker that controls building enclosure controller amplifier failed. Didn't have a spare, down for a night because couldn't move enclosure building. Back up the next day (worked a solution until spare was received and put into place). Hope won't be switching back and forth between generator and utility for testing/burning off fuel too much more.

Telescope is working as expected. Same for 0.5m. Instrumentation: cold weather in January led to some issues with some of the instruments (primarily with shutters). KOSMOS slit viewer shutter was sticky when below its temp rating. Replaced with spare, looking to get more spares on hand. Disperser and mask wheels also struggling in cold weather. Diffuser mechanism also something that hoping to schedule repair during next engineering time. Agile is back and we're working with them on path forward (reinstall as is or have them try to repair further, programmable gain still not functioning). Rest of the instruments have all been behaving nominally. Spec had issue with detector reading zeros but reset of camera controller did the trick.

Misty: long-term solution to avoiding cold issues? IN light of extreme weather and future. BK: When install instrumentation or spec new things, try to spec operating temp to -20 C. If we get down to -15C that's our de facto closure limit but there are some parts that we can't get manufactures to give us an extended cold temperature range. Hard to retrofit or find components if they don't have them. The -15C limit is due to telescope itself having issues. Not something that we've spent a lot of additional time to find solutions for more extreme conditions during winter.

We have some unassigned time in Q1 (mostly B halves). Feb 08, 17, 18 - have some requests already but if there are any urgent requests or high priority (e.g. students) otherwise will allocate the time to ppl who already made requests. More time in March - one fraction A half but the rest is B.

This time is listed as OPEN on the Q1 schedule. Users should look at the schedule and follow the usual channels for requesting this time (i.e. email Russet, Amanda, Ben, Nancy and their institutional scheduler when submitting requests, and provide a proposal cover page if you don't already have a program scheduled for the current quarter).

The 2025 Q1 ARCSAT scheduled is posted. We only had 2 requests this quarter (note added in proof: we received a third request, so there are now 3 programs scheduled to use ARCSAT in Q1). We are continuing to struggle with frequent failures in communication with the camera (which manifests as a “support library error”); FlareCam is getting old. The temporary fix is fairly quick so if observers can keep an eye on it and report errors it can be recovered pretty quickly. In early March we will have an onsite class and other stuff going on so ARCSAT is not available during weeks when the schedule is marked BUSY.

There is a plan to run a test tomorrow for converting the site to new servers (but this may be postponed due to illness).

The IT team at APO is still researching applications to improve password handling; all we know right now is that the switching of the TUI password is coming.

We have heard no news from NSF regarding the pre-proposal that was submitted for ASPEN; we hope to hear soon. Sarah reported that since having received the Ocotillo CDR report on December 2 her team has been evaluating the comments and preparing a response. They also applied for some funding and are waiting to hear back. It is clear that the instrument will come together in a phased approach, bringing the three channels online one at a time. They will start with the IFU and follow up with MOS component, which is more costly and mechanically complicated. The hope is that this would enable supported data reduction development along the way and troubleshooting as we go. It also spaces out the cost, which reduces impact on the overall observatory budget. Their goal is to have their response report back to the CDR review panel by the end of the month.

Nancy is working on developing a document that lays out the decommissioning plan for DIS. The instrument has not been serviced and will only be available in Q1 (and Q2) in a shared risk mode.

Open action items from previous meetings:

• Nancy: get ROM costs for new KOSMOS grisms. Sarah reported that she has a finalized quote from one vendor. The grism design turns out to be a little complicated, but also, costs have gone up, so these are more expensive than we expected (ballpark $20k apiece). Therefore, we need to explore potential funding sources. We have reached an impasse in terms of prioritizing the desired grism; Nancy will amass all previous emails with grism requests and send to UC reps to query users about whether there is broad interest in a particular grism. This could help identify funding sources, either individually or collaboratively. STATUS: OPEN. Nancy has started a proposal to the Mt. Cuba Foundation for several grisms and hopes to have it submitted before the next Users Committee meeting.
• Nancy: Ensure that a short description of the impact of satellites be posted on the APO website somewhere and updated every 6 months or so. Nancy received examples of this issue from Anne Verbiscer and will get something added to the current (and new) web site. STATUS: OPEN.
• UC reps: remind your users to review the Q1 schedule and request OPEN/DD time if they can make a good case for it. STATUS: OPEN.

Open action items from this meeting:

• None.

None

The next meeting will be on February 4, 2025 at 10:30 MST.