Tag: jupyterhub

python 3.x – PySpark on Jupyterhub K8s || Unable to query data || Class org.apache.hadoop.fs.s3a.S3AFileSystem not found

Pyspark Version: 2.4.5 Hive Version: 1.2 Hadoop Version: 2.7 AWS-SDK Jar: 1.7.4 Hadoop-AWS: 2.7.3 When I am trying to show data I am getting Class org.apache.hadoop.fs.s3a.S3AFileSystem not found while I am passing all the information which all are required. I passed all three for this config fs.s3.aws.credentials.provider org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider com.amazonaws.auth.InstanceProfileCredentialsProvider com.amazonaws.auth.EnvironmentVariableCredentialsProvider…

Continue Reading python 3.x – PySpark on Jupyterhub K8s || Unable to query data || Class org.apache.hadoop.fs.s3a.S3AFileSystem not found

Jupyter WebIO in binder – JupyterHub

I just learned how to run julia notebooks in binder. Now I want to use the PlotlyJS package in julia. Usually, when I need a package in julia on binder, I install it locally in a conda environment first and then copy Project.toml and Manifest.toml to my repository to start…

Continue Reading Jupyter WebIO in binder – JupyterHub

Jupyter notebook not loading the correct virtual environment

We deployed Jupyterhub in Kubernetes using the helm chart , the chart version is 0.9.0 github.com/jupyterhub/zero-to-jupyterhub-k8s/tree/main/jupyterhub zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/installation.html Users creating virtual environments using the terminal and created kernals as well, But when they accessing the jupyter notebook using the newly created kernal and run the command !conda info It is not…

Continue Reading Jupyter notebook not loading the correct virtual environment

How to set NotebookApp.iopub_data_rate_limit and others NotebookApp settings in JupyterHub?

Open the command line and enter jupyter notebook –NotebookApp.iopub_data_rate_limit=1e10 This should start jupyter with the increased data rate. You have to create config file using this command $ jupyter notebook –generate-config, The answer in this link 04: “IOPUB data rate exceeded” problem of “jupyter low memory..” in windows: open cmd:…

Continue Reading How to set NotebookApp.iopub_data_rate_limit and others NotebookApp settings in JupyterHub?

handle VPN drops for long-running notebooks : JupyterNotebooks

Summary of my needs Retail supply-chain data from relational databases. Data sets are usually 1 – 100 GB in size. All data is on-prem in my employer’s data center, not cloud based. Pandas or Dask and scikit-learn for clustering, classification, and regression Models often take several hours to train. Pandas…

Continue Reading handle VPN drops for long-running notebooks : JupyterNotebooks

How to upgrade Kernel version? – JupyterHub

I have.a JupyterHub running in a local K8s cluster, deployed via KubeApps. It works like.a charm, except for the fact that it is running Python 3.7. I would like to upgrade it to Python 3.10. I tried to run conda update python and conda install -y python=3.10 from the notebook,…

Continue Reading How to upgrade Kernel version? – JupyterHub

How to actually use scopes – JupyterHub

tim April 24, 2022, 3:35pm #1 Hey Guys,at the moment I am writing my bachelor thesis, part of it is JupyterHub. In the docs they stated that RBAC is replacing full admin access – seems clear to me. My only question is: If I grant one account a ‘servers’ permission,…

Continue Reading How to actually use scopes – JupyterHub

docker compose – How to enable authentication via openLDAP on JupyterHub without SSL certificate?

I am trying to create a JupyterHub that uses an LDAP to authenticate users. The JupyterHub is working but when I am trying log on the web page show a Error 500. I must clarify that when I tried it with the PAM method it worked without problems, but when…

Continue Reading docker compose – How to enable authentication via openLDAP on JupyterHub without SSL certificate?

What’s new in DataSpell 2022.1

JupyterHub, a platform that provides multi-user access to remote Jupyter servers, received a major upgrade with version 2.0, which introduced roles and scopes. DataSpell 2022.1 has adopted the API changes of the new version and allows connections to JupyterHub 2.0 by URL, the same way connections work for Jupyter and…

Continue Reading What’s new in DataSpell 2022.1

How to link GCP VM disks allocated by dynamic pvc-HEXHEXHEX and `claim-username` again after irrevocably losing all kubernetes yaml? – Zero to JupyterHub on Kubernetes

TL;DR I destroyed my claim-usernames on a prod cluster deployed using current z2jh, and have a mob of nervous scientists scared they lost their data… how do I go about rebinding a hundred disk-HEXHEX GCP VM disks to the appropriate jupyterhub users? In the process of finally deploying / converting…

Continue Reading How to link GCP VM disks allocated by dynamic pvc-HEXHEXHEX and `claim-username` again after irrevocably losing all kubernetes yaml? – Zero to JupyterHub on Kubernetes

Filtering outgoing container traffic – General Discussions

melhayany (Melhayany) April 12, 2022, 10:12pm #1 Hello Docker Community, I am facing a networking issue with my application to run JupyterHub using Docker containers. To give you a bit of context. I created a virtual machine, on it: port 2022 (ssh) served by my server, port 443 (https) served…

Continue Reading Filtering outgoing container traffic – General Discussions

Jupyter Community Forum

Sign Up Log In Log In Popular Jupyterlab on top of the hub and k8sZero to JupyterHub on Kubernetes Unable to use javascript magic functionZero to JupyterHub on Kubernetes New Jupyter Distinguished ContributorsMeta Welcome Wayne as a new forum moderatorMeta Ipywidgets 7.7.0 releasedWidgets NumFOCUS Job Boardjobs Jupyter-videochat: Video Chat with…

Continue Reading Jupyter Community Forum

Jupyterhub openshift with no S2I capability – Zero to JupyterHub on Kubernetes

Hi All I have searched lot in google but couldn’t find answer.we have openshift infrastructure available, but only limited functionalities.in that we’re not allowed to use S2I but only able to boost container via jenkins and run oc command there(jenkinsfile).problem we have now is we can’t even download jupyterhub image…

Continue Reading Jupyterhub openshift with no S2I capability – Zero to JupyterHub on Kubernetes

JupyterHub apiUrl detection fails when hub.baseUrl is set

The Jupyterhub helm chart includes a hub.baseUrl for serving the hub under a path other than /, e.g. /jupyter/. I believe this breaks the automatic API setting at github.com/dask/dask-gateway/blob/7d1659db8b2122d1a861ea820a459fd045fc3f02/resources/helm/dask-gateway/templates/gateway/configmap.yaml#L97. I’m not sure if the baseUrl is available where that code is running, so it may be hard to detect the…

Continue Reading JupyterHub apiUrl detection fails when hub.baseUrl is set

Python JupyterHub Developer at Pachyderm

Posted on: 03/22/2022 Location: (REMOTE) full time Crunchbase | Original Source Tags: jupyter github ml typescript docker kubernetes aws javascript python **About Pachyderm** At Pachyderm, we’re building an open-source enterprise-grade data science platform that lets you deploy and manage multi-stage, language-agnostic data pipelines while maintaining complete reproducibility and provenance. If…

Continue Reading Python JupyterHub Developer at Pachyderm

Tutorial: How to Connect Jupyter Notebooks to Ocean for Apache Spark – The Spot by NetApp Blog

Jupyter Notebook is a web-based interactive computational environment for creating notebook documents. It supports programming languages – such as Python, Scala, R – and is largely used for data engineering, data analysis, machine learning, and further interactive, exploratory computing. Think of notebooks like a developer console or terminal, but with…

Continue Reading Tutorial: How to Connect Jupyter Notebooks to Ocean for Apache Spark – The Spot by NetApp Blog

Install Jupyter Notebook on Debian 11 Bullseye Linux

Tutorial to learn the steps involve in the installation of Jupyter Notebook on Debian 11 Bullseye using command terminal. What is Jupyter Notebook? The Jupyter Project is a non-profit initiative that aims to develop and provide open-source software and open standards for interactive work. One of the most famous products…

Continue Reading Install Jupyter Notebook on Debian 11 Bullseye Linux

CORS issues behind nginx ingress after updating to JupyterHub 2.1.1 – JupyterHub

Hi there, I noticed multiple issues after upgrading to JupyterHub 2.1.1 from 2.0.0. I stopped seeing spawn progress when starting the singleuser server. Logs: [I 2022-03-04 20:57:47.436 JupyterHub log:189] 302 GET /hub/spawn -> /hub/spawn-pending/konstantin.taletskiy@labshare.org (konstantin.taletskiy@labshare.org@192.168.15.108) 1006.02ms [I 2022-03-04 20:57:47.524 JupyterHub pages:405] konstantin.taletskiy@labshare.org is pending spawn [I 2022-03-04 20:57:47.528 JupyterHub log:189]…

Continue Reading CORS issues behind nginx ingress after updating to JupyterHub 2.1.1 – JupyterHub

JupyterLab Installation and Configuration – HEAVY.AI Docs

container_name: omnisciserver image: omnisci/omnisci-ee-cuda:v5.9.0 – /var/lib/omnisci:/omnisci-storage – /var/lib/omnisci/data/omnisci_import/jhub_omnisci_dropbox:/jhub_omnisci_dropbox # If using binary encryption, uncomment the below line to override the default # command that uses startomnisci, noting that you must have an existing OmniSci # data directory and run initdb before making this change. # command: /bin/bash -c “/omnisci/bin/omnisci_web_server –config…

Continue Reading JupyterLab Installation and Configuration – HEAVY.AI Docs

UH Manoa Campus Events Calendar

Scientific Workflows and Gateways February 18, 2:00pm – 4:00pm Mānoa Campus, In Person at 103 Keller Hall, UH Manoa or Online via Zoom Workshop Description Sciences are becoming more data intensive and thus managing, handling and exploring data are becoming more prevalent issues in research. Using these data sets…

Continue Reading UH Manoa Campus Events Calendar

How can I connect to remote MSSQL server from jupyterHub notebook using windows authentication instead of MSSQL users? – Zero to JupyterHub on Kubernetes

I want to connect to Microsft SQL server database from jupyternotebook which is based on kubernetes z2jH installation, I made a cutome notebook image which has pyodbc & odbc inside, and could connect to MSSQL DB with sql user; however, it would be much easier to connect via Windows user…

Continue Reading How can I connect to remote MSSQL server from jupyterHub notebook using windows authentication instead of MSSQL users? – Zero to JupyterHub on Kubernetes

[jupyterhub/binderhub] Your connection isn’t private: https://hub-binder.mybinder.ovh/

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs: If you haven’t done so already, check out Jupyter’s Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more…

Continue Reading [jupyterhub/binderhub] Your connection isn’t private: https://hub-binder.mybinder.ovh/

JupyterHub on Host Network with HTTPS failing with 503: Service Unavailable – Zero to JupyterHub on Kubernetes

Hi All, Hope everyone is well! I have been working on a small project for an internal network to get JupyterHub running. I have gotten pretty far in the development, but now stuck on getting HTTPS working with the host network option. JupyterHub is running on internal VMs and thus…

Continue Reading JupyterHub on Host Network with HTTPS failing with 503: Service Unavailable – Zero to JupyterHub on Kubernetes

UC San Diego’s Jupyterhub Platform Aids Students With Data-Intensive Computing Needs | Computing

Jan. 24, 2022 — In basic UC San Diego trend, an overheard dialog at a campus espresso cart has changed into an interdisciplinary mission that’s making computing-intensive coursework extra thrilling whereas saving properly over $1 million {dollars} thus far. The effort offers UC San Diego graduate and undergraduate college students—and…

Continue Reading UC San Diego’s Jupyterhub Platform Aids Students With Data-Intensive Computing Needs | Computing

Jupyterhub – nltk – unable to use stopwords – resource stopwords not found ( Python, Nltk )

Problem : ( Scroll to solution ) I am using below code to use stopwords through jupyter notebook. I have hosted jupyter on Linux server and using the notebook. python3 -m nltk.downloader stopwords python3 -m nltk.downloader words python3 -m nltk.downloader punkt python3 >>>from nltk.corpus import stopwords >>>stop_words = set(stopwords.words(“english”)) >>>print(stop_words)…

Continue Reading Jupyterhub – nltk – unable to use stopwords – resource stopwords not found ( Python, Nltk )

apache spark – Pyspark kernel in jupyterhub, access to master remotely

i have two kernels for Spark, one to run locally and one to run towards a cluster. Is there a way to set an environment variable to my spark master so that users dont have to define master in SparkContext in the kernel which is to speak with the spark…

Continue Reading apache spark – Pyspark kernel in jupyterhub, access to master remotely

Python through jupyterhub. Simple animation with Xarray data

I’m working on a project where I will try to do an animation of the mixed layer depth for 12 months. My current concern is that I want to do an animation where I only show the current month with data in the plot, but they keep showing vertically in…

Continue Reading Python through jupyterhub. Simple animation with Xarray data

python – Where to upload the image files to JupyterHub server (ltjh)

I would like to create a jupyter notebook that displays some images like this. In the case of my computer, the image files are stored in the local device. So I wrote one-line tag to show the image. <img src=”images/python_with_Birds.gif” width=”400″/> But if I want to share my created jupyter…

Continue Reading python – Where to upload the image files to JupyterHub server (ltjh)

make Pyspark working inside jupyterhub

You need to configure the pyspark kernel. On my server jupyter kernels are located at: /usr/local/share/jupyter/kernels/ You can create a new kernel by making a new directory: mkdir /usr/local/share/jupyter/kernels/pyspark Then create the kernel.json file – I paste my as a reference: { “display_name”: “pySpark (Spark 1.6.0)”, “language”: “python”, “argv”: […

Continue Reading make Pyspark working inside jupyterhub

Intern with UCSB IT | UC Santa Barbara Information Technology

All of our UCSB IT internships are now full. Please check back for future internship opportunities.  UCSB IT offers student interns a paid opportunity to complement their academic training by learning practical skills in a high demand field. UCSB IT values our students’ opinions and believes their ideas drive the successful…

Continue Reading Intern with UCSB IT | UC Santa Barbara Information Technology

okay to run docker build test only on changes to Dockerfiles ?

I’d like to see a quicker response from the CI system when we make a pull request etc. I’ve opened this issue to ask if you agree to only run the job testing the build of the Dockerfiles when we change the Dockerfiles. Would that be reasonable? If so, I’ll…

Continue Reading okay to run docker build test only on changes to Dockerfiles ?

how do you connect a javascript app to a jupyterhub kernel? – Python jupyterhub

We want to embed a notebook type thing in our app but @nteract/notebook-app-component is not finished, jupyterlab is heavy and takes over the whole screen, thebelab is also heavy (1.6Mb) … these solutions don’t work for a simple web app to embed a notebook kernel right now How do you…

Continue Reading how do you connect a javascript app to a jupyterhub kernel? – Python jupyterhub

Moodle Authentication for JupyterHub Docker image | Moodle | Docker | Python | GitLab | OAuth

I have a JupyterHub installation managed within Docker. Also, I have a Moodle installation. I need a freelancer who is competent in both JupyterHub configuration and Moodle configuration. The task is to take my GitLab repository of the JupyterHub Docker file as a basis, create a new branch and do…

Continue Reading Moodle Authentication for JupyterHub Docker image | Moodle | Docker | Python | GitLab | OAuth

Error in Authenticator.pre_spawn_start: CalledProcessError Command – jupyterhub

Hello all, I am new to JupyterHub and trying to integrate it with LTI as an authenticator. We have our own course provider platform like edX and Canvas so we are doing custom configuration in the jupyterhub_config.py. However, I am partially successful in doing so because when I launch my…

Continue Reading Error in Authenticator.pre_spawn_start: CalledProcessError Command – jupyterhub

Jupyterhub 2.0.0 c.Spawner.notebook_dir vs $HOME for setting custom home directory – JupyterHub

I just upgraded JupyterHub v1.3.0 to v2.0.0 on Linux CentOS 8 (I know this OS is EOL soon, planning to migrate to Ubuntu 20.04LTS…) following instructions here, but I am now experiencing difficulties setting custom home directories. I previously set up user home directories on an NFS mounted file path,…

Continue Reading Jupyterhub 2.0.0 c.Spawner.notebook_dir vs $HOME for setting custom home directory – JupyterHub

github – Source control or Project folder connections with Jupyter

I need help in deciding the easiest approach to centralized the storage of all project-related files such as data extracts and python code which can then be accessed and executed into JypterHub without manually uploading into the JypterHub folder Can I store the updated version of the .ipynb file and…

Continue Reading github – Source control or Project folder connections with Jupyter

windows – Cannot stop jupyterhub in Docker?

So, I have Docker installed on Windows 10. Then, in cmd.exe, I run a docker container: C:Usersuser>docker start -i image_name … containing an Ubuntu 20.04 image, where, once started, from the command line I run: $ jupyterhub -f /etc/jupyterhub/jupyterhub_config.py So, this has been running for three days without a problem;…

Continue Reading windows – Cannot stop jupyterhub in Docker?

[jupyterhub/jupyterhub] Empty Admin page when base_url is set

Bug description If JupyterHub.base_url is set in jupyterhub_config.py, visitig the Admin tab on jupyterhub v.2.0.0 shows the empty page. Expected behaviour Without setting JupyterHub.base_url the Admin page displays the list of users: Actual behaviour The page shows only the version number: I see some errors in the log: I 2021-12-10…

Continue Reading [jupyterhub/jupyterhub] Empty Admin page when base_url is set

JupyterHub & IRkernel misbehaving — next steps for debugging? – JupyterHub

I am trying to get an R kernel working in Jupyterhub — it was working on an Ubuntu 16.04 install but it stopped when I upgraded to 20.04 and did a fresh install of everything (default proxy and processor setups; no docker complexities; installed through pip). I followed the instructions…

Continue Reading JupyterHub & IRkernel misbehaving — next steps for debugging? – JupyterHub

Jupyterhub – iframe in canvas LMS

I have two different servers with one Canvas LMS installed and one JupyterHub. I include as an external app with LTI JupyterHub in Canvas and setting the configuration: c.JupyterHub.tornado_settings = { ‘headers’: { ‘Content-Security-Policy’: “frame-ancestors ‘self’ xxxxxxx.yyy” }} he problem is that when I click on the “start my server”…

Continue Reading Jupyterhub – iframe in canvas LMS

How to open file from JupyterHub workspace when using %%javascript magic?

I use JupyterHub in combination with JupyterLab. I have a user “test” and a file “data.csv” in the workspace of that user. I would like to use %%javascript magic and load that file with d3.js. However, the url “./data.csv” does not work. =>What is the right relative url to load…

Continue Reading How to open file from JupyterHub workspace when using %%javascript magic?

jupyter lab – How do I set logo and launchers in Jupyterlab and Jupyterhub?

For the logo, I am not sure about JupyterHub-specific solutions, but you could see how Elyra does that for their JupyterLab distribution with Elyra theme extension here (under Apache License, so it should not be pasted in here). In short they first locate the icon in the “top” area of…

Continue Reading jupyter lab – How do I set logo and launchers in Jupyterlab and Jupyterhub?

Problem connecting singleuser pod to spark – Zero to JupyterHub on Kubernetes

I’m having an issue connecting to Spark from a singleuser pod launched via Jupyterhub 2.0. I suspect it’s a networking issue. I can launch a dedicated jupyterlab pod and connect to the Spark instance; I need to set the spark.driver.host to the IP addr of the pod, but once done,…

Continue Reading Problem connecting singleuser pod to spark – Zero to JupyterHub on Kubernetes

jupyter – JupyterHub populate user home with notebooks (and data)

I have a JupyterHub server hosted on an Ubuntu machine. (Mainly) since I want to avoid adding hundreds of potential users to the machine I opted to use c.JupyterHub.authenticator_class=”jupyterhub.auth.DummyAuthenticator” (with a password) The Spawner I use is c.JupyterHub.spawner_class=”jupyterhub.spawner.SimpleLocalProcessSpawner” although I also tried ‘jupyterhub.spawner.LocalProcessSpawner’. I want users to have some pre-populated…

Continue Reading jupyter – JupyterHub populate user home with notebooks (and data)

Spawn timeouts on autoscaling – Zero to JupyterHub on Kubernetes

Hey folks, my team is using the Jupyterhub Helm Chart (v0.11.1) with the userScheduler enabled and we’re noticing timeouts due to pods having to wait for large images to finish pulling. This seems to only happen when two notebooks are spawned in rapid succession and when the first of said…

Continue Reading Spawn timeouts on autoscaling – Zero to JupyterHub on Kubernetes

Repo template to use a Linux Desktop Application on a binder / jupyterhub

Generate a Git repository that can run any Linux GUI application in the browser via mybinder.org or any JupyterHub from this template repository! Uses jupyter-remote-desktop-proxy to work. How to use this reop 1. Create a new repo using this as a template Use the Use this template button on GitHub….

Continue Reading Repo template to use a Linux Desktop Application on a binder / jupyterhub

docker – JupyterHub “400 : Bad Request OAuth state missing from cookies” generic authentication

I am trying to deploy a jupyterhub service behind a NGINX reverse proxy on OpenStack and using the generic authentication class to authenticate users from an external OIDC provider. After redirecting from the authentication server I get a “400: Bad Request OAuth state missing from cookies” error message. Here are…

Continue Reading docker – JupyterHub “400 : Bad Request OAuth state missing from cookies” generic authentication

How to make created custom (conda) environments persist across user sessions in Jupyter Hub on AWS EKS cluster – Zero to JupyterHub on Kubernetes

Hi All, Greetings!! Hope everyone is doing well. Thanks in advance. I have installed the Jupyter hub on the AWS EKS cluster. Objective 1: We want to allow users to create conda environments using Jupyter lab UI which needs to be persisted across all user sessions meaning user 1 has…

Continue Reading How to make created custom (conda) environments persist across user sessions in Jupyter Hub on AWS EKS cluster – Zero to JupyterHub on Kubernetes