Tag: jupyterhub
python 3.x – PySpark on Jupyterhub K8s || Unable to query data || Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
Pyspark Version: 2.4.5 Hive Version: 1.2 Hadoop Version: 2.7 AWS-SDK Jar: 1.7.4 Hadoop-AWS: 2.7.3 When I am trying to show data I am getting Class org.apache.hadoop.fs.s3a.S3AFileSystem not found while I am passing all the information which all are required. I passed all three for this config fs.s3.aws.credentials.provider org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider com.amazonaws.auth.InstanceProfileCredentialsProvider com.amazonaws.auth.EnvironmentVariableCredentialsProvider…
Jupyter WebIO in binder – JupyterHub
I just learned how to run julia notebooks in binder. Now I want to use the PlotlyJS package in julia. Usually, when I need a package in julia on binder, I install it locally in a conda environment first and then copy Project.toml and Manifest.toml to my repository to start…
Jupyter notebook not loading the correct virtual environment
We deployed Jupyterhub in Kubernetes using the helm chart , the chart version is 0.9.0 github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/main/jupyterhub zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/installation.html Users creating virtual environments using the terminal and created kernals as well, But when they accessing the jupyter notebook using the newly created kernal and run the command !conda info It is not…
How to set NotebookApp.iopub_data_rate_limit and others NotebookApp settings in JupyterHub?
Open the command line and enter jupyter notebook –NotebookApp.iopub_data_rate_limit=1e10 This should start jupyter with the increased data rate. You have to create config file using this command $ jupyter notebook –generate-config, The answer in this link 04: “IOPUB data rate exceeded” problem of “jupyter low memory..” in windows: open cmd:…
handle VPN drops for long-running notebooks : JupyterNotebooks
Summary of my needs Retail supply-chain data from relational databases. Data sets are usually 1 – 100 GB in size. All data is on-prem in my employer’s data center, not cloud based. Pandas or Dask and scikit-learn for clustering, classification, and regression Models often take several hours to train. Pandas…
How to upgrade Kernel version? – JupyterHub
I have.a JupyterHub running in a local K8s cluster, deployed via KubeApps. It works like.a charm, except for the fact that it is running Python 3.7. I would like to upgrade it to Python 3.10. I tried to run conda update python and conda install -y python=3.10 from the notebook,…
How to actually use scopes – JupyterHub
tim April 24, 2022, 3:35pm #1 Hey Guys,at the moment I am writing my bachelor thesis, part of it is JupyterHub. In the docs they stated that RBAC is replacing full admin access – seems clear to me. My only question is: If I grant one account a ‘servers’ permission,…
docker compose – How to enable authentication via openLDAP on JupyterHub without SSL certificate?
I am trying to create a JupyterHub that uses an LDAP to authenticate users. The JupyterHub is working but when I am trying log on the web page show a Error 500. I must clarify that when I tried it with the PAM method it worked without problems, but when…
What’s new in DataSpell 2022.1
JupyterHub, a platform that provides multi-user access to remote Jupyter servers, received a major upgrade with version 2.0, which introduced roles and scopes. DataSpell 2022.1 has adopted the API changes of the new version and allows connections to JupyterHub 2.0 by URL, the same way connections work for Jupyter and…
How to link GCP VM disks allocated by dynamic pvc-HEXHEXHEX and `claim-username` again after irrevocably losing all kubernetes yaml? – Zero to JupyterHub on Kubernetes
TL;DR I destroyed my claim-usernames on a prod cluster deployed using current z2jh, and have a mob of nervous scientists scared they lost their data… how do I go about rebinding a hundred disk-HEXHEX GCP VM disks to the appropriate jupyterhub users? In the process of finally deploying / converting…
Filtering outgoing container traffic – General Discussions
melhayany (Melhayany) April 12, 2022, 10:12pm #1 Hello Docker Community, I am facing a networking issue with my application to run JupyterHub using Docker containers. To give you a bit of context. I created a virtual machine, on it: port 2022 (ssh) served by my server, port 443 (https) served…
Jupyter Community Forum
Sign Up Log In Log In Popular Jupyterlab on top of the hub and k8sZero to JupyterHub on Kubernetes Unable to use javascript magic functionZero to JupyterHub on Kubernetes New Jupyter Distinguished ContributorsMeta Welcome Wayne as a new forum moderatorMeta Ipywidgets 7.7.0 releasedWidgets NumFOCUS Job Boardjobs Jupyter-videochat: Video Chat with…
Jupyterhub openshift with no S2I capability – Zero to JupyterHub on Kubernetes
Hi All I have searched lot in google but couldn’t find answer.we have openshift infrastructure available, but only limited functionalities.in that we’re not allowed to use S2I but only able to boost container via jenkins and run oc command there(jenkinsfile).problem we have now is we can’t even download jupyterhub image…
JupyterHub apiUrl detection fails when hub.baseUrl is set
The Jupyterhub helm chart includes a hub.baseUrl for serving the hub under a path other than /, e.g. /jupyter/. I believe this breaks the automatic API setting at github.com/dask/dask-gateway/blob/7d1659db8b2122d1a861ea820a459fd045fc3f02/resources/helm/dask-gateway/templates/gateway/configmap.yaml#L97. I’m not sure if the baseUrl is available where that code is running, so it may be hard to detect the…
Python JupyterHub Developer at Pachyderm
Posted on: 03/22/2022 Location: (REMOTE) full time Crunchbase | Original Source Tags: jupyter github ml typescript docker kubernetes aws javascript python **About Pachyderm** At Pachyderm, we’re building an open-source enterprise-grade data science platform that lets you deploy and manage multi-stage, language-agnostic data pipelines while maintaining complete reproducibility and provenance. If…
Tutorial: How to Connect Jupyter Notebooks to Ocean for Apache Spark – The Spot by NetApp Blog
Jupyter Notebook is a web-based interactive computational environment for creating notebook documents. It supports programming languages – such as Python, Scala, R – and is largely used for data engineering, data analysis, machine learning, and further interactive, exploratory computing. Think of notebooks like a developer console or terminal, but with…
Install Jupyter Notebook on Debian 11 Bullseye Linux
Tutorial to learn the steps involve in the installation of Jupyter Notebook on Debian 11 Bullseye using command terminal. What is Jupyter Notebook? The Jupyter Project is a non-profit initiative that aims to develop and provide open-source software and open standards for interactive work. One of the most famous products…
CORS issues behind nginx ingress after updating to JupyterHub 2.1.1 – JupyterHub
Hi there, I noticed multiple issues after upgrading to JupyterHub 2.1.1 from 2.0.0. I stopped seeing spawn progress when starting the singleuser server. Logs: [I 2022-03-04 20:57:47.436 JupyterHub log:189] 302 GET /hub/spawn -> /hub/spawn-pending/konstantin.taletskiy@labshare.org (konstantin.taletskiy@labshare.org@192.168.15.108) 1006.02ms [I 2022-03-04 20:57:47.524 JupyterHub pages:405] konstantin.taletskiy@labshare.org is pending spawn [I 2022-03-04 20:57:47.528 JupyterHub log:189]…
JupyterLab Installation and Configuration – HEAVY.AI Docs
container_name: omnisciserver image: omnisci/omnisci-ee-cuda:v5.9.0 – /var/lib/omnisci:/omnisci-storage – /var/lib/omnisci/data/omnisci_import/jhub_omnisci_dropbox:/jhub_omnisci_dropbox # If using binary encryption, uncomment the below line to override the default # command that uses startomnisci, noting that you must have an existing OmniSci # data directory and run initdb before making this change. # command: /bin/bash -c “/omnisci/bin/omnisci_web_server –config…
UH Manoa Campus Events Calendar
Scientific Workflows and Gateways February 18, 2:00pm – 4:00pm Mānoa Campus, In Person at 103 Keller Hall, UH Manoa or Online via Zoom Workshop Description Sciences are becoming more data intensive and thus managing, handling and exploring data are becoming more prevalent issues in research. Using these data sets…
How can I connect to remote MSSQL server from jupyterHub notebook using windows authentication instead of MSSQL users? – Zero to JupyterHub on Kubernetes
I want to connect to Microsft SQL server database from jupyternotebook which is based on kubernetes z2jH installation, I made a cutome notebook image which has pyodbc & odbc inside, and could connect to MSSQL DB with sql user; however, it would be much easier to connect via Windows user…
[jupyterhub/binderhub] Your connection isn’t private: https://hub-binder.mybinder.ovh/
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs: If you haven’t done so already, check out Jupyter’s Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more…
JupyterHub on Host Network with HTTPS failing with 503: Service Unavailable – Zero to JupyterHub on Kubernetes
Hi All, Hope everyone is well! I have been working on a small project for an internal network to get JupyterHub running. I have gotten pretty far in the development, but now stuck on getting HTTPS working with the host network option. JupyterHub is running on internal VMs and thus…
UC San Diego’s Jupyterhub Platform Aids Students With Data-Intensive Computing Needs | Computing
Jan. 24, 2022 — In basic UC San Diego trend, an overheard dialog at a campus espresso cart has changed into an interdisciplinary mission that’s making computing-intensive coursework extra thrilling whereas saving properly over $1 million {dollars} thus far. The effort offers UC San Diego graduate and undergraduate college students—and…
Jupyterhub – nltk – unable to use stopwords – resource stopwords not found ( Python, Nltk )
Problem : ( Scroll to solution ) I am using below code to use stopwords through jupyter notebook. I have hosted jupyter on Linux server and using the notebook. python3 -m nltk.downloader stopwords python3 -m nltk.downloader words python3 -m nltk.downloader punkt python3 >>>from nltk.corpus import stopwords >>>stop_words = set(stopwords.words(“english”)) >>>print(stop_words)…
apache spark – Pyspark kernel in jupyterhub, access to master remotely
i have two kernels for Spark, one to run locally and one to run towards a cluster. Is there a way to set an environment variable to my spark master so that users dont have to define master in SparkContext in the kernel which is to speak with the spark…
Python through jupyterhub. Simple animation with Xarray data
I’m working on a project where I will try to do an animation of the mixed layer depth for 12 months. My current concern is that I want to do an animation where I only show the current month with data in the plot, but they keep showing vertically in…
python – Where to upload the image files to JupyterHub server (ltjh)
I would like to create a jupyter notebook that displays some images like this. In the case of my computer, the image files are stored in the local device. So I wrote one-line tag to show the image. <img src=”images/python_with_Birds.gif” width=”400″/> But if I want to share my created jupyter…
make Pyspark working inside jupyterhub
You need to configure the pyspark kernel. On my server jupyter kernels are located at: /usr/local/share/jupyter/kernels/ You can create a new kernel by making a new directory: mkdir /usr/local/share/jupyter/kernels/pyspark Then create the kernel.json file – I paste my as a reference: { “display_name”: “pySpark (Spark 1.6.0)”, “language”: “python”, “argv”: […
Intern with UCSB IT | UC Santa Barbara Information Technology
All of our UCSB IT internships are now full. Please check back for future internship opportunities. UCSB IT offers student interns a paid opportunity to complement their academic training by learning practical skills in a high demand field. UCSB IT values our students’ opinions and believes their ideas drive the successful…
okay to run docker build test only on changes to Dockerfiles ?
I’d like to see a quicker response from the CI system when we make a pull request etc. I’ve opened this issue to ask if you agree to only run the job testing the build of the Dockerfiles when we change the Dockerfiles. Would that be reasonable? If so, I’ll…
how do you connect a javascript app to a jupyterhub kernel? – Python jupyterhub
We want to embed a notebook type thing in our app but @nteract/notebook-app-component is not finished, jupyterlab is heavy and takes over the whole screen, thebelab is also heavy (1.6Mb) … these solutions don’t work for a simple web app to embed a notebook kernel right now How do you…
Moodle Authentication for JupyterHub Docker image | Moodle | Docker | Python | GitLab | OAuth
I have a JupyterHub installation managed within Docker. Also, I have a Moodle installation. I need a freelancer who is competent in both JupyterHub configuration and Moodle configuration. The task is to take my GitLab repository of the JupyterHub Docker file as a basis, create a new branch and do…
Error in Authenticator.pre_spawn_start: CalledProcessError Command – jupyterhub
Hello all, I am new to JupyterHub and trying to integrate it with LTI as an authenticator. We have our own course provider platform like edX and Canvas so we are doing custom configuration in the jupyterhub_config.py. However, I am partially successful in doing so because when I launch my…
Jupyterhub 2.0.0 c.Spawner.notebook_dir vs $HOME for setting custom home directory – JupyterHub
I just upgraded JupyterHub v1.3.0 to v2.0.0 on Linux CentOS 8 (I know this OS is EOL soon, planning to migrate to Ubuntu 20.04LTS…) following instructions here, but I am now experiencing difficulties setting custom home directories. I previously set up user home directories on an NFS mounted file path,…
github – Source control or Project folder connections with Jupyter
I need help in deciding the easiest approach to centralized the storage of all project-related files such as data extracts and python code which can then be accessed and executed into JypterHub without manually uploading into the JypterHub folder Can I store the updated version of the .ipynb file and…
windows – Cannot stop jupyterhub in Docker?
So, I have Docker installed on Windows 10. Then, in cmd.exe, I run a docker container: C:Usersuser>docker start -i image_name … containing an Ubuntu 20.04 image, where, once started, from the command line I run: $ jupyterhub -f /etc/jupyterhub/jupyterhub_config.py So, this has been running for three days without a problem;…
[jupyterhub/jupyterhub] Empty Admin page when base_url is set
Bug description If JupyterHub.base_url is set in jupyterhub_config.py, visitig the Admin tab on jupyterhub v.2.0.0 shows the empty page. Expected behaviour Without setting JupyterHub.base_url the Admin page displays the list of users: Actual behaviour The page shows only the version number: I see some errors in the log: I 2021-12-10…
JupyterHub & IRkernel misbehaving — next steps for debugging? – JupyterHub
I am trying to get an R kernel working in Jupyterhub — it was working on an Ubuntu 16.04 install but it stopped when I upgraded to 20.04 and did a fresh install of everything (default proxy and processor setups; no docker complexities; installed through pip). I followed the instructions…
Jupyterhub – iframe in canvas LMS
I have two different servers with one Canvas LMS installed and one JupyterHub. I include as an external app with LTI JupyterHub in Canvas and setting the configuration: c.JupyterHub.tornado_settings = { ‘headers’: { ‘Content-Security-Policy’: “frame-ancestors ‘self’ xxxxxxx.yyy” }} he problem is that when I click on the “start my server”…
How to open file from JupyterHub workspace when using %%javascript magic?
I use JupyterHub in combination with JupyterLab. I have a user “test” and a file “data.csv” in the workspace of that user. I would like to use %%javascript magic and load that file with d3.js. However, the url “./data.csv” does not work. =>What is the right relative url to load…
jupyter lab – How do I set logo and launchers in Jupyterlab and Jupyterhub?
For the logo, I am not sure about JupyterHub-specific solutions, but you could see how Elyra does that for their JupyterLab distribution with Elyra theme extension here (under Apache License, so it should not be pasted in here). In short they first locate the icon in the “top” area of…
Problem connecting singleuser pod to spark – Zero to JupyterHub on Kubernetes
I’m having an issue connecting to Spark from a singleuser pod launched via Jupyterhub 2.0. I suspect it’s a networking issue. I can launch a dedicated jupyterlab pod and connect to the Spark instance; I need to set the spark.driver.host to the IP addr of the pod, but once done,…
jupyter – JupyterHub populate user home with notebooks (and data)
I have a JupyterHub server hosted on an Ubuntu machine. (Mainly) since I want to avoid adding hundreds of potential users to the machine I opted to use c.JupyterHub.authenticator_class=”jupyterhub.auth.DummyAuthenticator” (with a password) The Spawner I use is c.JupyterHub.spawner_class=”jupyterhub.spawner.SimpleLocalProcessSpawner” although I also tried ‘jupyterhub.spawner.LocalProcessSpawner’. I want users to have some pre-populated…
Spawn timeouts on autoscaling – Zero to JupyterHub on Kubernetes
Hey folks, my team is using the Jupyterhub Helm Chart (v0.11.1) with the userScheduler enabled and we’re noticing timeouts due to pods having to wait for large images to finish pulling. This seems to only happen when two notebooks are spawned in rapid succession and when the first of said…
Repo template to use a Linux Desktop Application on a binder / jupyterhub
Generate a Git repository that can run any Linux GUI application in the browser via mybinder.org or any JupyterHub from this template repository! Uses jupyter-remote-desktop-proxy to work. How to use this reop 1. Create a new repo using this as a template Use the Use this template button on GitHub….
docker – JupyterHub “400 : Bad Request OAuth state missing from cookies” generic authentication
I am trying to deploy a jupyterhub service behind a NGINX reverse proxy on OpenStack and using the generic authentication class to authenticate users from an external OIDC provider. After redirecting from the authentication server I get a “400: Bad Request OAuth state missing from cookies” error message. Here are…
How to make created custom (conda) environments persist across user sessions in Jupyter Hub on AWS EKS cluster – Zero to JupyterHub on Kubernetes
Hi All, Greetings!! Hope everyone is doing well. Thanks in advance. I have installed the Jupyter hub on the AWS EKS cluster. Objective 1: We want to allow users to create conda environments using Jupyter lab UI which needs to be persisted across all user sessions meaning user 1 has…