Differences

This shows you the differences between two versions of the page.

--- uc:uc2026 [2026/07/06 16:32] – holtz
+++ uc:uc2026 [2026/08/06 22:42] (current) – nchanove
@@ Line 1: / Line 1: @@
+===== APO 3.5-m Users Committee Meeting, 8/04/2026 =====
+Attending: Nancy Chanover (NMSU), Aleksandr Mosenkov (BYU), Anne Verbiscer (UVa), Moire Prescott (NMSU), Michael Hayden (OU), Sarah Tuttle (UW), Bill Ketzeback (APO), Adam Kowalski (CU), Eric Bellm (UW), Ben Williams (UW)
+=== User feedback and comments from institutional representatives ===
+  * BYU - nothing to report
+  * JHU - nothing to report (sent by email)
+  * UVa - nothing to report
+  * NMSU - nothing to report
+  * Colgate - no report
+  * Williams - no report
+  * Colorado - a user inquired about the possibility of time trades with WIYN. Adam will put the user in touch with Nancy to discuss it further.
+  * Wyoming - no report
+  * Oklahoma - nothing to report
+  * Washington - nothing to report
+  * Georgia State - no report
+----
+=== Telescope and Instruments Report ===
+The detailed site report is included below, followed by additional information discussed during today's meeting.
+// 3.5-m Telescope and Instruments Highlights, 07/08/26 – 08/04/26 //
+) Overview
+Monsoon weather was the cause of the majority of the unscheduled downtime for July. The National Weather Service designates the official start of the monsoon season in New Mexico as June 15, running through September 30, though the actual climatological onset of rains varies by region across the state and Arizona. The start of increased moisture at the observatory commenced on July 8th with brief to heavy rain showers each day or evening. While we have been enjoying the increased moisture, we have not experienced the forecasted super-El Niño. On Saturday afternoon of July 11th, there was strong evidence that the 3.5m enclosure experienced a direct lightning strike, causing some equipment damage or temporarily affecting several systems requiring system restarts.
+During the three weeks leading up to the start of summer shutdown activities, there were a few telescope-reported troubles. Besides one night when the motion errors for the altitude axis were particularly bothersome during checkout, there were no other reports related to slews. A single problem with the rotator cleared by rebooting the controller. The lightning strike, mentioned in the previous paragraph, affected telescope positions and loss of communications to the tertiary controller and some damage to the enclosure door limits.
+A few notable instrument issues were recurring during this reporting cycle. In addition to the telescope issues from the lightning strike, other systems affected included dewpoint sensors and the all-sky camera.
+Instruments experiencing issues this month were primarily ARCTIC and ECHELLE. Details for each are below.
+Summer shutdown was another shortened maintenance period lasting two weeks. Activities are detailed below.
+The LVM2 (ARC Property) spectrographs were remediated, cleared, and removed from the Dunn Solar Telescope High Bay during this time and relocated to their new home at McDonald Observatory.
+The 2.5m Telescope has an opening for either a telescope operations specialist or a support astronomer. The positions differ primarily based on education and experience. The postings close at midnight on the 7th of August. The links to those postings can be found at:
+[[https://careers.nmsu.edu/jobs/telescope-operations-specialist-las-cruces-new-mexico-united-states]]
+[[https://careers.nmsu.edu/jobs/support-astronomer-remote-locations-new-mexico-united-states]]
+There were no on-site training classes or visiting instruments this reporting cycle.
+The APO house trailer is no longer available for occupancy. NMSU and APO are discussing its future, but the loss of three dorm rooms may affect class visits in the future.
+) Operations
+.5m Telescope: The telescope has been performing nominally, with a few reported minor issues. There were multiple reported altitude motion errors, but all on a single afternoon, while performing test slews. The observing specialist was able to eventually recover, and no on-sky time was lost.  One power brownout during the day resulted in automatic mirror covers needing to be reset. Telescope systems affected by the lightning strike mentioned earlier were able to be recovered, with the exception of permanent damage to the enclosure door limit switches. Repair parts are on order. For shutdown, the primary, secondary, and tertiary mirrors all underwent an aqueous cleaning. The altitude and azimuth encoders were cleaned, and many of the various system fans were replaced or cleaned to improve airflow.
+.5m Telescope: Dcam and UVAcam are both available options for users. The software (and possible computer hardware) still seem unstable, requiring multiple reboots and application restarts.
+KOSMOS: The system is cold and stable. The instrument is performing nominally.
+ARCTIC: The diffuser rotation mechanism is still unreliable. The mechanism that moves the diffuser in and out of the optical path is still functional in the meantime. The lower-left quadrant of the CCD has shown intermittent elevated noise or complete saturation in bias frames. This problem has progressively gotten worse over the past two months. These seem related to elevated dome temperatures. Over shutdown, we greatly increased airflow to the controller, but there seemed to be no effect. We experimented with swapping the video board in the controller, also with no improvement. With no support from Astronomical Research Cameras (Bob Leech has sunsetted the company), this may be a difficult problem to solve. Users are cautioned to carefully inspect their data, especially their calibration frames. We may be forced to switch to single amplifier readout for science observations while we continue to troubleshoot. Spares are nearly non-existent.
+ARCES: Vacuum servicing was performed on the instrument in July. The instrument is cold and usable, but we will not know if the improvements to the interorder light ratios, a measurement of contamination along the light path, will degrade rapidly or not.
+NICFPS: System is cooled and usable.
+TripleSpec: System is cooled and usable. A new instrument block was performed on sky as part of shutdown recovery to improve pointing and offsets.
+APOLLO: The instrument is usable for laser ranging. Maintaining the instrument's temperature during extreme environmental conditions remains challenging. Laser power adjustments were done as part of summer shutdown recovery.
+Agile/ SoonerCam: Agile has been decommissioned. The instrument rotator is still not performing nominally, and we will continue troubleshooting to prepare for SoonerCam. Parts to create a dummy weight for testing and modifications to the instrument filter wheel are in the planning stage. This was one of the planned shutdown tasks that were deferred due to unexpected summer machining delays and to other instrument work. SoonerCam-ICC software development has begun. Gordon continues to report good progress in understanding the GPS timing card software API.
+DIS: The instrument has been decommissioned. It has been removed from the observing floor and will eventually have expensive and critical parts removed before the rest is scrapped. The removal of the instrument from the observing floor frees floor space for the future delivery of Ocotillo.
+----
+=== Additional site, telescope and instruments discussion ===
+Both ARCTIC and ARCES are currently having issues. ARCES was just vacuum serviced; there is something in the cryostat (we have suspected this for many years due to the observed contamination). Our new vacuum pump allowed us to pull more gas out of chamber, during which time a strong volatile smell was identified as coming from inside the instrument. This is probably something inside the vacuum that is deteriorating. The inter-order light (IOL) after cool-down dropped really rapidly; after an inadvertent partial warm up last night, the IOL improved briefly, indicating that the contaminant migrated from near the CCD to a colder region of cryostat. If the IOL continues to drop we may have to pull it offline and put it back into servicing. [Note added in proof: the IOL has reached the point of recommended servicing so the decision was made to begin that process on Friday August 7. Affected users have been notified.]
+Regarding ARCTIC, for the past few months while in multi-amplifier mode the lower left quadrant has been excessively noisy. Recently it went from being saturated on the edge of quadrant to fully saturated. At first this was believed to be a thermal effect; we worked on it over summer shutdown, swapped out some electronics, and improved airflow over controller. Coming out of shutdown, we recommend that it be used only in single amplifier mode. This was attempted last night but that showed a new problem, namely other parts of detector were showing a similar effect. Cary is troubleshooting this further. We have limited spares and can't get more due the sunsetting of the company that manufactured the camera control system, so we are watching this carefully.
+Everything else is performing well. Agile and DIS have been decommissioned. APOLLO had some laser power adjustments. KOSMOS is cold and stable. Coming out of shutdown there were a few issues with the 3.5m telescope but those have been stable the last few days and hopefully can be addressed during the next engineering time.
+Shutdown itself went very well. The site report above contains the latest updates concerning the situation at Sunspot.
+Although this is generally a good practice, in light of the aforementioned instrument issues we recommend that Q4 proposers try to identify backup programs that would make use of a different instrument. Most importantly, observers should inspect their data as they are being acquired as this is the fastest way for the observatory staff to learn about and address problems that may arise.
+----
+=== 2026 Q3 3.5m scheduling ===
+The Q3 schedule is posted. There are several slots of OPEN and DD time starting in mid-August. Users should submit requests for this time to the scheduling team (Amanda Townsend, Russet McMillan, Ben Williams, and Nancy Chanover), and cc their institutional scheduler. [Note added in proof: we may have some additional open time due to the upcoming ARCES servicing so anyone who has lost time due to weather in recent months should submit requests if they can use it.]
+=== 2026 Q4 3.5m scheduling ===
+The Q4 institutional allocations were sent out yesterday; proposals are due on August 25. In light of the anticipated on-site housing crunch due to the loss of the housing trailer at Sunspot, it is //imperative// that anyone planning a class visit in Q4 should notify the scheduling team ASAP so we can understand the need for on-site accommodations.
+=== 2026 Q3 0.5 scheduling ===
+The ARCSAT schedule is posted and ARCSAT is fully booked for Q3.
+----
+=== PypeIt update ===
+As reported in last month's minutes, Jon Holtzman has added modules for KOSMOS and TripleSpec into the development branch of PypeIt. If there are any users that would be interested in trying those, they can do so if they download this version (e.g., see [[https://pypeit.readthedocs.io/en/stable/installing.html#development-or-pre-release-versions]], they would want the develop branch). If anyone has any questions they can contact Jon directly. These should be included in a release version relatively soon; it would be great to get some test users even sooner!
+Nancy reiterated the need for a user group devoted to software testing (at the moment, especially for KOSMOS and TripleSpec users to try out these PypeIt modules). Adam K indicated that he is willing to help. UC reps: please ask your users (and/or work with your institutional schedulers to identify your KOSMOS and TripleSpec users) to help out with this testing and to have those users communicate with Jon Holtzman about their experiences.
+----
+=== New instrument initiatives ===
+  * KOSMOS new grism - the new quote arrived and we will be placing the order shortly.
+  * Ocotillo - We now have all of the optics in hand and just ordered the bulk of mechanical parts. They will arrive in approximately 3 weeks and then assembly will begin. The TEG team is now working on software, specifically to ensure that there is infrastructure in place to ensure communication between the 3 channels even if only one is functional first.
+  * ASPEN - We received the report from the Preliminary Design Review team, which was pretty favorable. The team received some good suggestions and is evaluating next steps. We are moving forward with the Atmospheric Dispersion Corrector, which will be installed and used with ARCES until ASPEN is built.
+  * SoonerCam - Gordon reports that he is able to control the timer card, he has become familiarized with the API, he can control the camera and read out FITS files. He is working on the time match and external triggering interface; it works using a unique feature of a timing card or with similar Agile heartbeat signal, one implementation is less complex than the other. He would like to assemble a focus group of potential SoonerCam users to better understand the envisioned use cases (e.g. what calibration frames would be needed, other design requirements) so he can finalize C++ library for imaging. UC reps: please notify your users that we are looking for some time domain / occultation / former Agile users to take part in this focus group. Any interested parties should let Nancy know so she can assemble this group.
+----
+=== Pilot data archive with NOIRLab ===
+We are moving forward with an arrangement to have ARC 3.5m data ingested into the NOIRLab data archive. We will pilot this first with a single instrument (likely ARCES), which will allow us to work out any kinks. The decision was made to have this be an OPT-OUT arrangement, i.e. ARCES observations will automatically be sent over to NOIRLab (where they will remain not broadly accessible until after a TBD proprietary period) unless a user decides to opt out. Please direct all questions and comments to Nancy.
+----
+=== ACTION ITEMS ===
+Open action items from previous meetings:
+  * Nancy will ensure that the final report from the Future of ARCSAT Working Group is made available to the Users Committee and put on the wiki. Status: OPEN (pending receipt of final version of report; still awaiting one appendix. Nancy said she will distribute the remainder of the report now even without the appendix).
+  *  APO: update text on TUI downloads page to reflect new procedure for downloading TUI. Status: PENDING; needs verification
+  * UC reps: notify your users about OPEN time available in June. Status: CLOSED.
+  * Nancy: follow-up on PypeIt implementation and how it handles KOSMOS reduction complexities. Status: OPEN.
+  * Nancy: Post job ad links in July meeting minutes. Status: CLOSED.
+New action items from this meeting:
+  * Nancy: revisit idea of time trades with WIYN.
+  * UC reps: identify KOSMOS and/or TripleSpec users in your department who would be willing to help test out the new PypeIt modules for those instruments. Have those users report their engagement with Jon Holtzman.
+  * UC reps: identify time domain / former Agile users who are willing to serve on SoonerCam focus group.
+  * UC reps: notify your users about OPEN time available in August-September.
+----
+=== All Other Business ===
+None.
+----
+=== Next Meeting ===
+The next Users Committee meeting will be on Tuesday September 1 at 10:30 MDT.
+----
+===== APO 3.5-m Users Committee Meeting, 7/07/2026 =====
+Attending: Chip Kobulnicky (UWy), Bill Ketzeback (APO), Misty Bentz (GSU), Kevin Schlaufman (JHU), Aleksandr Mosenkov (BYU), Adam Kowalski (CU), Eric Bellm (UW), Russet McMillan (APO), Ben Williams (UW)
+=== User feedback and comments from institutional representatives ===
+  * Washington - nothing to report
+  * Wyoming - nothing to report
+  * Georgia State - nothing to report
+  * BYU - nothing to report
+  * Oklahoma - no report
+  * UVa - nothing to report (by email)
+  * Colorado - nothing to report
+  * JHU - nothing to report
+  * Williams - no report
+  * Colgate - no report
+  * NMSU - nothing to report (by email)
+----
+=== Telescope and Instruments Report ===
+The detailed site report is included below, followed by additional information discussed during today's meeting.
+// 3.5-m Telescope and Instruments Highlights, 06/03/26 – 07/07/26 //
+) Overview
+Job ads are out; APO is hiring for a new SDSS observer:
+[[https://careers.nmsu.edu/jobs/telescope-operations-specialist-las-cruces-new-mexico-united-states]]. Please encourage any interested candidates to apply.
+The weather in June had a fair number of clear or partially clear nights for observing. We did see some precipitation on the nights that were clouded out. Several small wildfires were sparked by lightning and quickly contained this past week. These wildfires were located within 50 miles of the observatory and could have posed a threat of smoke.
+There were minimal telescope-reported troubles in June other than an occasional motion error during afternoon checkout.
+A few notable instrument issues occurred during this reporting cycle. Instruments experiencing issues this month were ARCTIC and ECHELLE. Details for each are below.
+Since the last Users Committee meeting, there were no visiting instruments. There was one on-site training class over the July 4th holiday weekend.
+The APO house trailer is no longer available for occupancy. NMSU and APO are discussing its future but the loss of three dorm rooms may affect class visits in the future.
+) Operations
+.5m Telescope: The telescope has been performing nominally, with a few minor issues reported. There were two reported altitude motion errors, but no on-sky time was lost because these occurred during afternoon test slews. One power brownout during the day resulted in the automatic mirror covers needing to be reset.
+.5m Telescope: Dcam and UVAcam are both available options for users. We experienced numerous TCS-related issues with the telescope only wanting to slew in a single direction per axis even when commanded to go the opposite direction. This error caused the instrument to run away until it reached a limit. It seemed suspiciously temperature-related, as it sometimes went away when we were troubleshooting. The computer was removed, cleaned, and all cards and cable connectors were reseated. It is currently available in a shared risk mode.
+KOSMOS: System is cold and stable. Instrument is performing nominally.
+ARCTIC: The diffuser rotation mechanism is still unreliable. We manufactured some new parts, but have not had the opportunity to install and test them. The mechanism that moves the diffuser in and out of the optical path is still functional in the meantime. The lower-left quadrant of the CCD has shown intermittent elevated noise or complete saturation in bias frames. These seem related to elevated dome temperatures. They seem to appear for a few exposures at the start of the night and then clear on their own. Users are cautioned to carefully inspect their data, especially their calibration frames.
+ARCES: We experienced a few failures again this month in the thorium argon lamp power supply, resulting in lamp dropouts or weak lines. We are working to source a spare power supply, but finding a distributor has been difficult. If identified by an echelle user during their cals, the observing specialists have procedures to identify which is occurring and adjust the power supply or move the mirror by hand. Users are advised to carefully examine their calibration frames. Interorder light ratios have decreased but remain rather good, so we will attempt to delay servicing as long as possible. Hopefully, we can delay until the July shutdown in two weeks.
+NICFPS: System is cooled and usable.
+TripleSpec: System is cooled and usable.
+APOLLO: The instrument is usable for laser ranging. Maintaining the instrument's temperature during extreme environmental conditions remains challenging.
+Agile/ SoonerCam: Agile has been decommissioned. The Agile instrument rotator is still not performing nominally, and we will continue troubleshooting to prepare for SoonerCam. Parts to create a dummy weight for testing and modifications to the instrument filter wheel are in the planning stage. SoonerCam parts are on hand, assembled, and software development has begun. Software development is in the early stages of learning the Hamamatsu and Brandywine software APIs. Gordon reported good progress in understanding the GPS timing card software API.
+DIS: The instrument has been decommissioned. It has been removed from the observing floor and will eventually have the expensive and critical parts removed before the rest is scrapped.
+----
+=== Additional site, telescope and instruments discussion ===
+See above, particularly the job postings:
+[[https://careers.nmsu.edu/jobs/telescope-operations-specialist-las-cruces-new-mexico-united-states]]
+----
+=== 2026 Q3 3.5m scheduling ===
+The Q3 schedule is posted. As a reminder, the 3.5m summer shutdown is scheduled for July 20 - August 2. The remaining OPEN or DD time is on July 11th (dark A half), 12th (short dark slot in B-half), July 17A, August 4B (short slot; right after summer shutdown). Users should submit requests for this time to the scheduling team (Amanda Townsend, Russet McMillan, Ben Williams, and Nancy Chanover), and cc their institutional scheduler.
+=== 2026 Q2 0.5 scheduling ===
+The ARCSAT schedule will be posted shortly. Users will be scheduled starting after summer shutdown.
+----
+=== Call for papers/grants info ===
+Nancy issued her annual call for scientific papers and student theses/dissertations that make use of data acquired with the 3.5m and/or ARCSAT. Please send her all such papers (accepted or published) covering Jan. 1 2025 to present no later than July 31, 2026. New this year: Nancy is also collecting information concerning funding awards that have been enabled through access to the 3.5m. Please send her the following information for any APO-related grants: PI name, Funding Agency/Program, Project Title, Total Budget (if willing to share it), period of performance (years only), and relevant 3.5m instrument(s). This information will not be made public, but will be summarized for the ARC Board.
+----
+=== PypeIt update ===
+Jon Holtzman reports that modules for KOSMOS and TripleSpec have been added into the develop branch of PypeIt. If there are any users that would be interested in trying those, they can do so if they download this version (e.g., see [[https://pypeit.readthedocs.io/en/stable/installing.html#development-or-pre-release-versions]], they would want the develop branch). If anyone has any questions they can contact Jon directly. These should be included in a release version relatively soon; it would be great to get some test users even sooner!
+Kevin asked if ARCES going to be added?  Bill said yes, that is the plan.
+Adam stated that there are lots of KOSMOS data reduction issues (for example, bleed-over and non-fixed pattern noise). How does PypeIt handle these, as users have needed to do this with custom software? We will report back on this at a future meeting.
+----
+=== New instrument initiatives ===
+  * Ocotillo - no update.
+  * ASPEN - Work on the Atmospheric Dispersion Corrector is proceeding as it will be used with ARCES even before ASPEN is completed. The ASPEN Preliminary Design Review was held in June and the instrument team is awaiting the final report from the review team. Three ASPEN-related papers will be presented at the SPIE meeting in Copenhagen this week.
+  * SoonerCam: see site report for progress related to SoonerCam and the Agile rotator.
+----
+=== ACTION ITEMS ===
+Open action items from previous meetings:
+  * Nancy will ensure that the final report from the Future of ARCSAT Working Group is made available to the Users Committee and put on the wiki. Status: OPEN (pending receipt of final version of report).
+  *  APO: update KOSMOS user manual web page regarding issue of flats bleeding into overscan region. Status: CLOSED.
+  *  Nancy: include grant information in upcoming call for annual update on papers resulting from 3.5m/ARCSAST. Status: CLOSED.
+  * APO: update scheduling webpage to replace Postscript calendars with PDF versions. Status: CLOSED.
+  * APO: update text on TUI downloads page to reflect new procedure for downloading TUI. Status: PENDING; needs verification.
+  * UC reps: notify your users about OPEN time available in June. Status: CLOSED
+New action items from this meeting:
+  * Nancy: follow-up on PypeIt implementation and how it handles KOSMOS reduction complexities.
+  * Nancy: Post job ad links in minutes.
+----
+=== All Other Business ===
+None.
+----
+=== Next Meeting ===
+The next Users Committee meeting will be on Tuesday August 4 at 10:30 MDT.
+----
 ===== APO 3.5-m Users Committee Meeting, 6/02/2026 =====