Introduction

In this article we're going to be taking a look at the try/except clause, and specifically how you can catch multiple exceptions in a single line, as well as how to use the suppress() method.

Both of these techniques will help you in writing more accessible and versatile code that adheres to DRY (don't repeat yourself) principles.

Let's start by looking at the problem:

try:
    do_the_thing()
except TypeError as e:
    do_the_other_thing()
except KeyError as e:
    do_the_other_thing()
except IndexError as e:
    do_the_other_thing()

Brutal.

As we can see, this is very WET code, we repeat the same invocation multiple times. Practices like this can make our code's reading and refactoring a living nightmare.

Rather than writing exceptions one after another, wouldn't it be better to group all of these exception handlers into a single line?

Multiple Exceptions

If you're just here for a quick answer, it's simple: use a tuple.

All the errors contained in the exception line tuple will be evaluated together:

try:
    do_the_thing()
except (TypeError, KeyError, IndexError) as e:
    do_the_other_thing()

Easy, right?

Avoiding Bad Practices

"Errors should never pass silently." - The Zen of Python.

try/except clauses are probably the most misused pattern in Python.

Used improperly, they end up being the cliché of drunks and lampposts, being used only when the Python interpreter start caroling the "12 Errors of Christmas".

It's very tempting to just put a try and a bare exception on a problem to "make it go away". By doing that, we're effectively sweeping the exceptions under the rug, which is a shame, especially since they can be wonderfully helpful in recovering from potentially fatal errors, or shining a light on hidden bugs.

That's why when using except clauses you should always be sure to specify the errors you know you could encounter, and exclude the ones you don't.

Letting your program fail is okay, even preferred, to just pretending the problem doesn't exist.

"Errors should never pass silently... unless explicitly silenced."

However, once in a blue moon when you do get the opportunity to ignore exception handling, you can use suppress():

from contextlib import suppress

with suppress(TypeError, KeyError, IndexError):
    do_the_thing()

The suppress() method takes a number of exceptions as its argument, and performs a try/except/pass with those errors. As you can see it also let's you write multiple exceptions in a single line.

This let's you avoid writing a try/except/pass manually:

try:
    do_the_thing()
except (TypeError, KeyError, IndexError) as e:
    pass

Better yet, it's also standard in any version of Python 3.4 and above!

Conclusion

In this article, we've covered how to handle multiple exceptions in a single line. We've also briefly gone over some bad practices of ignoring exceptions, and used the supress() function to supress exceptions explicitly.

Django 3.2 alpha 1 is now available. It represents the first stage in the 3.2 release cycle and is an opportunity for you to try out the changes coming in Django 3.2.

Django 3.2 has a mezcla of new features which you can read about in the in-development 3.2 release notes.

This alpha milestone marks the feature freeze. The current release schedule calls for a beta release in about a month and a release candidate about a month from then. We'll only be able to keep this schedule if we get early and often testing from the community. Updates on the release schedule are available on the django-developers mailing list.

As with all alpha and beta packages, this is not for production use. But if you'd like to take some of the new features for a spin, or to help find and fix bugs (which should be reported to the issue tracker), you can grab a copy of the alpha package from our downloads page or on PyPI.

The PGP key ID used for this release is Carlton Gibson: E17DF5C82B4F9D00

When you have a recording where one or more people are talking, it's useful to have a highly accurate and automated way to extract the spoken words into text. Once you have the text, you can use it for further analysis or as an accessibility feature.

In this tutorial, we'll use a high accuracy speech-to-text web application programming interface called AssemblyAI to extract text from an MP3 recording (many other formats are supported as well).

With the code from this tutorial, you will be able to take an audio file that contains speech such as this example one I recorded and output a highly accurate text transcription like this:

An object relational mapper is a code library that automates the transfer of 
data stored in relational, databases into objects that are more commonly used
in application code or EMS are useful because they provide a high level 
abstraction upon a relational database that allows developers to write Python 
code instead of sequel to create read update and delete, data and schemas in 
their database. Developers can use the programming language. They are 
comfortable with to work with a database instead of writing SQL...

(the text goes on from here but I abbreviated it at this point)

Tutorial requirements

Throughout this tutorial we are going to use the following dependencies, which we will install in just a moment. Make sure you also have Python 3, preferably 3.6 or newer installed, in your environment:

We will use the following dependencies to complete this tutorial:

• requests version 2.24.0 to make HTTP requests to the AssemblyAI speech-to-text API
• An AssemblyAI account, which you can sign up for a free API access key here

All code in this blog post is available open source under the MIT license on GitHub under the transcribe-speech-text-script directory of the blog-code-examples repository. Use the source code as you desire for your own projects.

Setting up the development environment

Change into the directory where you keep your Python virtual environments. I keep mine in a subdirectory named venvs within my user's home directory. Create a new virtualenv for this project using the following command.

python3 -m venv ~/venvs/pytranscribe

Activate the virtualenv with the activate shell script:

source ~/venvs/pytranscribe/bin/activate

After the above command is executed, the command prompt will change so that the name of the virtualenv is prepended to the original command prompt format, so if your prompt is simply $, it will now look like the following:

(pytranscribe) $

Remember, you have to activate your virtualenv in every new terminal window where you want to use dependencies in the virtualenv.

We can now install the requests package into the activated but otherwise empty virtualenv.

pip install requests==2.24.0

Look for output similar to the following to confirm the appropriate packages were installed correctly from PyPI.

(pytranscribe) $ pip install requests==2.24.0
Collecting requests==2.24.0
  Using cached https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests==2.24.0)
  Using cached https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests==2.24.0)
  Using cached https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl
Collecting chardet<4,>=3.0.2 (from requests==2.24.0)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<3,>=2.5 (from requests==2.24.0)
  Using cached https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl
Installing collected packages: certifi, urllib3, chardet, idna, requests
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 requests-2.24.0 urllib3-1.25.10

We have all of our required dependencies installed so we can get started coding the application.

Uploading, initiating and transcribing audio

We have everything we need to start building our application that will transcribe audio into text. We're going to build this application in three files:

1. upload_audio_file.py: uploads your audio file to a secure place on AssemblyAI's service so it can be access for processing. If your audio file is already accessible with a public URL, you don't need to do this step, you can just follow this quickstart
2. initiate_transcription.py: tells the API which file to transcribe and to start immediately
3. get_transcription.py: prints the status of the transcription if it is still processing, or displays the results of the transcription when the process is complete

Create a new directory named pytranscribe to store these files as we write them. Then change into the new project directory.

mkdir pytranscribe
cd pytranscribe

We also need to export our AssemblyAI API key as an environment variable. Sign up for an AssemblyAI account and log in to the AssemblyAI dashboard, then copy "Your API token" as shown in this screenshot:

export ASSEMBLYAI_KEY=your-api-key-here

Note that you must use the export command in every command line window that you want this key to be accessible. The scripts we are writing will not be able to access the API if you do not have the token exported as ASSEMBLYAI_KEY in the environment you are running the script.

Now that we have our project directory created and the API key set as an environment variable, let's move on to writing the code for the first file that will upload audio files to the AssemblyAI service.

Uploading the audio file for transcription

Create a new file named upload_audio_file.py and place the following code in it:

import argparse
import os
import requests


API_URL = "https://api.assemblyai.com/v2/"


def upload_file_to_api(filename):
    """Checks for a valid file and then uploads it to AssemblyAI
    so it can be saved to a secure URL that only that service can access.
    When the upload is complete we can then initiate the transcription
    API call.
    Returns the API JSON if successful, or None if file does not exist.
    """
    if not os.path.exists(filename):
        return None

    def read_file(filename, chunk_size=5242880):
        with open(filename, 'rb') as _file:
            while True:
                data = _file.read(chunk_size)
                if not data:
                    break
                yield data

    headers = {'authorization': os.getenv("ASSEMBLYAI_KEY")}
    response = requests.post("".join([API_URL, "upload"]), headers=headers,
                             data=read_file(filename))
    return response.json()

The above code imports the argparse, os and requests packages so that we can use them in this script. The API_URL is a constant that has the base URL of the AssemblyAI service. We define the upload_file_to_api function with a single argument, filename that should be a string with the absolute path to a file and its filename.

Within the function, we check that the file exists, then use Request's chunked transfer encoding to stream large files to the AssemblyAI API.

The os module's getenv function reads the API that was set on the command line using the export command with the getenv. Make sure that you use that export command in the terminal where you are running this script otherwise that ASSEMBLYAI_KEY value will be blank. When in doubt, use echo $ASSEMBLY_AI to see if the value matches your API key.

To use the upload_file_to_api function, append the following lines of code in the upload_audio_file.py file so that we can properly execute this code as a script called with the python command:

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("filename")
    args = parser.parse_args()
    upload_filename = args.filename
    response_json = upload_file_to_api(upload_filename)
    if not response_json:
        print("file does not exist")
    else:
        print("File uploaded to URL: {}".format(response_json['upload_url']))

The code above creates an ArgumentParser object that allows the application to obtain a single argument from the command line to specify the file we want to access, read and upload to the AssmeblyAI service.

If the file does not exist, the script will print a message that the file couldn't be found. In the happy path where we do find the correct file at that path, then the file is uploaded using the code in upload_file_to_api function.

Execute the completed upload_audio_file.py script by running it on the command line with the python command. Replace FULL_PATH_TO_FILE with an absolute path to the file you want to upload, such as /Users/matt/devel/audio.mp3.

python upload_audio_file.py FULL_PATH_TO_FILE

Assuming the file is found at the location that you specified, when the script finishes uploading the file, it will print a message like this one with a unique URL:

File uploaded to URL: https://cdn.assemblyai.com/upload/463ce27f-0922-4ea9-9ce4-3353d84b5638

This URL is not public, it can only be used by the AssemblyAI service, so no one else will be able to access your file and its contents except for you and their transcription API.

The part that is important is the last section of the URL, in this example it is 463ce27f-0922-4ea9-9ce4-3353d84b5638. Save that unique identifier because we need to pass it into the next script that initiates the transcription service.

Initiate transcription

Next, we'll write some code to kick off the transcription. Create a new file named initiate_transcription.py. Add the following code to the new file.

import argparse
import os
import requests


API_URL = "https://api.assemblyai.com/v2/"
CDN_URL = "https://cdn.assemblyai.com/"


def initiate_transcription(file_id):
    """Sends a request to the API to transcribe a specific
    file that was previously uploaded to the API. This will
    not immediately return the transcription because it takes
    a moment for the service to analyze and perform the
    transcription, so there is a different function to retrieve
    the results.
    """
    endpoint = "".join([API_URL, "transcript"])
    json = {"audio_url": "".join([CDN_URL, "upload/{}".format(file_id)])}
    headers = {
        "authorization": os.getenv("ASSEMBLYAI_KEY"),
        "content-type": "application/json"
    }
    response = requests.post(endpoint, json=json, headers=headers)
    return response.json()

We have the same imports as the previous script and we've added a new constant, CDN_URL that matches the separate URL where AssemblyAI stores the uploaded audio files.

The initiate_transcription function essentially just sets up a single HTTP request to the AssemblyAI API to start the transcription process on the audio file at the specific URL passed in. This is why passing in the file_id is important: that completes the URL of the audio file that we are telling AssemblyAI to retrieve.

Finish the file by appending this code so that it can be easily invoked from the command line with arguments.

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("file_id")
    args = parser.parse_args()
    file_id = args.file_id
    response_json = initiate_transcription(file_id)
    print(response_json)

Start the script by running the python command on the initiate_transcription file and pass in the unique file identifier you saved from the previous step.

# the FILE_IDENTIFIER is returned in the previous step and will
# look something like this: 463ce27f-0922-4ea9-9ce4-3353d84b5638
python initiate_transcription.py FILE_IDENTIFIER

The API will send back a JSON response that this script prints to the command line.

{'audio_end_at': None, 'acoustic_model': 'assemblyai_default', 'text': None, 
 'audio_url': 'https://cdn.assemblyai.com/upload/463ce27f-0922-4ea9-9ce4-3353d84b5638', 
 'speed_boost': False, 'language_model': 'assemblyai_default', 'redact_pii': False, 
 'confidence': None, 'webhook_status_code': None, 
 'id': 'gkuu2krb1-8c7f-4fe3-bb69-6b14a2cac067', 'status': 'queued', 'boost_param': None, 
 'words': None, 'format_text': True, 'webhook_url': None, 'punctuate': True, 
 'utterances': None, 'audio_duration': None, 'auto_highlights': False, 
 'word_boost': [], 'dual_channel': None, 'audio_start_from': None}

Take note of the value of the id key in the JSON response. This is the transcription identifier we need to use to retrieve the transcription result. In this example, it is gkuu2krb1-8c7f-4fe3-bb69-6b14a2cac067. Copy the transcription identifier in your own response because we will need it to check when the transcription process has completed in the next step.

Retrieving the transcription result

We have uploaded and begun the transcription process, so let's get the result as soon as it is ready.

How long it takes to get the results back can depend on the size of the file, so this next script will send an HTTP request to the API and report back the status of the transcription, or print the output if it's complete.

Create a third Python file named get_transcription.py and put the following code into it.

import argparse
import os
import requests


API_URL = "https://api.assemblyai.com/v2/"


def get_transcription(transcription_id):
    """Requests the transcription from the API and returns the JSON
    response."""
    endpoint = "".join([API_URL, "transcript/{}".format(transcription_id)])
    headers = {"authorization": os.getenv('ASSEMBLYAI_KEY')}
    response = requests.get(endpoint, headers=headers)
    return response.json()


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("transcription_id")
    args = parser.parse_args()
    transcription_id = args.transcription_id
    response_json = get_transcription(transcription_id)
    if response_json['status'] == "completed":
        for word in response_json['words']:
            print(word['text'], end=" ")
    else:
        print("current status of transcription request: {}".format(
              response_json['status']))

The code above has the same imports as the other scripts. In this new get_transcription function, we simply call the AssemblyAI API with our API key and the transcription identifier from the previous step (not the file identifier). We retrieve the JSON response and return it.

In the main function we handle the transcription identifier that is passed in as a command line argument and pass it into the get_transcription function. If the response JSON from the get_transcription function contains a completed status then we print the results of the transcription. Otherwise, print the current status which is either queued or processing before it is completed.

Call the script using the command line and the transcription identifier from the previous section:

python get_transcription.py TRANSCRIPTION_ID

If the service has not yet started working on the transcript then it will return queued like this:

current status of transcription request: queued

When the service is currently working on the audio file it will return processing:

current status of transcription request: processing

When the process is completed, our script will return the text of the transcription, like you see here:

An object relational mapper is a code library that automates the transfer of 
data stored in relational, databases into objects that are more commonly used
in application code or EMS are useful because they provide a high level 

...(output abbreviated)

That's it, we've got our transcription!

You may be wondering what to do if the accuracy isn't where you need it to be for your situation. That is where boosting accuracy for keywords or phrases and selecting a model that better matches your data come in. You can use either of those two methods to boost the accuracy of your recordings to an acceptable level for your situation.

What's next?

We just finished writing some scripts that call the AssemblyAI API to transcribe recordings with speech into text output.

Next, take a look at some of their more advanced documentation that goes beyond the basics in this tutorial:

• Supported file formats
• Transcribing dual channel/stereo recordings
• Getting speaker labels (speaker diarization)

Questions? Let me know via an issue ticket on the Full Stack Python repository, on Twitter @fullstackpython or @mattmakai. See something wrong with this post? Fork this page's source on GitHub and submit a pull request.

One of the common complaints about Python is that it is slow. There are languages and runtimes that can execute code faster, but they are not as easy to be productive with, so many people are willing to make that tradeoff. There are some use cases, however, that truly need the benefit of faster execution. To address this problem Kevin Modzelewski helped to create the Pyston intepreter that is focused on speeding up unmodified Python code. In this episode he shares the history of the project, discusses his current efforts to optimize a fork of the CPython interpreter, and his goals for building a business to support the ongoing work to make Python faster for everyone. This is an interesting look at the opportunities that exist in the Python ecosystem and the work being done to address some of them.

Hello geeks and welcome in this article, we will cover cv2 normalize(). Along with that, we will also look at its syntax for an overall better understanding. Then we will see the application of all the theory part through a couple of examples. The cv2 is a cross-platform library designed to solve all computer vision-related problems. We will look at its application and work later in this article. But first, let us look at the definition of the function.

In general, normalization means repeating data repetition and eliminate unwanted characteristics. So, Image normalization can be understood as to how we change an image’s pixel intensity. Often it is linked with increasing contrast, which helps in better image segmentation.

Syntax

cv.normalize(img,  norm_img)

This is the general syntax of our function. Here the term “img” represents the image file to be normalized. “Norm_img” represents the user’s condition to be implemented on the image. As we move ahead in this article, we will develop a better understanding of this function.

How Cv2 Normalize works?

We have discussed the definition and general syntax of Cv2 Normalize. In this section, we will try to get a brief idea about how it works. With the help of this, we can remove noise from an image. We bring the image in a range of intensity values, which makes the image less stressful and more normal to our senses. Primarily it does the job of making the subject image a bit clearer. It does so with the help of several parameters that we will discuss in detail in the next section.

Application of Cv2 Normalize

In this section, we will see what difference the cv2 Normalize code makes. To achieve this, we will first use the Cv2 imshow to display an image, after which we will use the normalize function and compare the 2 images to spot the difference.

import cv2

img = cv2.imread('3.jpeg',1)
cv2.imshow("sample",img)
cv2.waitKey(5000)

Output:

Here we have successfully used the imshow() function to print our image. As I have already covered the imshow() function, I will not go in-depth about it here. Our picture is not very clear, and its overall appearance can be improved considerably. Now let use our function and see the difference.

import cv2 as cv
import numpy as ppool
img = cv.imread("3.jpeg")
norm = ppool.zeros((800,800))
final = cv.normalize(img,  norm, 0, 255, cv.NORM_MINMAX)
cv.imshow('Normalized Image', final)
cv.imwrite('city_normalized.jpg', final)
cv.waitKey(5000)

Output:

See what our function does; the change is quite evident. When you compare it with the previous one, you can notice that it is far clearer and has better contrast.

Now let us try to decode and understand the code that helped us achieve it. Here at first, we have imported cv2. After which, we have imported the NumPy module. Then we have used the imread() function to read our image. After that, we have used the numpy function zeros, which gives a new array of 800*800. Then we have used the cv normalized syntax. Here 1st we have our image name, second normalization condition. Then we have 255, which is the upper limit of our array, which means values beyond that will not be stored in it. Then, at last, we have used cv.NORM_MINMAX, in this case, the lower value is alpha, and the higher value is beta, so the function works between them.

How to get the original image back?

Using the normalized function creates a separate new file for the subject image. Our original image remains unchanged, and hence to obtain it, we can use the imshow() function.

Conclusion

In this article, we covered the Cv2 normalize(). We looked at its syntax and example. We tried to understand what difference this function can make to your image through example. As in our case, by applying this, we were able to achieve a much clearer picture. In the end, we can conclude that cv2 normalize() helps us by changing the pixel intensity and increasing the overall contrast.

I hope this article was able to clear all doubts. But in case you have any unsolved queries feel free to write them below in the comment section.

The post CV2 Normalize() in Python Explained With Examples appeared first on Python Pool.

Hello geeks and welcome in today’s article, we will cover Python Syslog. Along with that, for proper understanding, we will look at different methods and also some sample code. Before moving that ahead let us first understand Syslog through its definition. The module provides an interface to the Unix Syslog library. Here Unix is an OS developed for multiuser and multitasking computers. A module named SysLogHandler is available in logging.handlers, a pure python library that can speak to the Syslog server.

Different Methods for Python Syslog

1. SYSLOG.SYLOG(MESSAGE,PRIORITY)

This function sends a string message to the system logger. Here logger keeps track of events when the software runs. Here the priority argument is optional and defaults to log_info, and it determines the priority.

2. SYSLOG.OPENLOG

This is used as an subsequent of syslog call. It takes an ident argument which of string type.

3. SYSLOG.CLOSELOG

This method is used for the purpose of resetting the syslog module.

4. SYSLOG.SETLOGMASK

This method is used to set up the priority mask to maskpri, It returns the previous mask value. When there is no priority maskpri is ignored.

Sample Code Covering Various Syslog Methods

In this section we will look at some sample codes in which we have will use the various methods discussed in the above section.

import syslog
import sys

syslog.openlog(sys.argv[0])

syslog.syslog(syslog.LOG_NOTICE, "notice-1")
syslog.syslog(syslog.LOG_NOTICE, "notice-2")

syslog.closelog()

Here above we can see the sample code. Here at first, we have imported the Syslog module. Along with that we have imported the sys which means system-specific parameters. Next, we have used openlog with a command sys. argv[0]. This command is a list that contains the command-line argument passed to the script. Next, we have a Syslog method and closed with a close log command.

SysLogHandler

As discussed at the start of the article it is a method available in logging. handlers. This particular method supports sending a message to a remote or Unix Syslog. Let us look at an example in order for a better understanding.

import logging
import logging.handlers
import sys

logger = logging.getLogger()
logger.setLevel(logging.INFO)
syslog = logging.handlers.SysLogHandler(address=("localhost", 8000))
logger.addHandler(syslog)
print (logger.handlers)

Output

Here at first, we have imported the logging module, an in-built module of python. Then we have imported logging handlers, which sends the log records to the appropriate destination. Next, we have imported SYS, as discussed above. Then in the next step, we have created an object with getlogger. Then in the next step, we have used the setlevel command. What it does is that all messages before this level are ignored. Then we have used our Sysloghandeler command. Next, we have used addHandler to add a specific handler to our logger “syslog” in our case. Finally, we have just used a print statement to print that handler.

Difference Between Syslog and Logging

This section will discuss the basic difference between the Syslog and logging in python. Here we have discussed Syslog in detail but before comparing the 2, let us look at the logging definition. It is an in-built module of python that helps the programmer keep track of events that are taking place. The basic difference between the 2 is that Syslog is more powerful, whereas the logging is easy and used for simple purposes. Another advantage of Syslog over logging is that it can send log lines to a different computer to have it logged there.

You might be interested in reading:

• Understanding Priority Queue in Python with Implementation
• Python is Not Recognized as an Internal or External Command
• Max Heap Python Implementation | Python Max Heap
• METHODS TO CONVERT TUPLE TO STRING IN PYTHON

Conclusion

In this article, we covered Python Syslog. We looked at its definition, use, and the different methods associated with it. In the end, we can conclude that it provides us with an interface of Unix.

I hope this article was able to clear all doubts. But in case you have any unsolved queries feel free to write them below in the comment section. Done reading this why not look at the FizzBuzz challenge next.

The post What is Python Syslog? Explained with Different methods appeared first on Python Pool.

Hello geeks, and welcome in this article, we will be covering “unhashable type: list.” It is a type of error that we come across when writing code in python. In this article, ur main objective is to look at this error. Along with that, we will also try to troubleshoot and get rid of this error. We will achieve all this with a couple of examples. But first, let us try to get a brief overview of why this error occurs.

Python dictionaries only accept hashable data-types as a key. Here the hashable data-types means those values whose value remains the same during the lifetime. But when we use the list data-type, which is non-hashable, we get this kind of error.

The error-“unhahasble type: list”

In this section, we will look at the reason due to which this error occurs. We will take into account everything discussed so far. Let us see this through an example:

numb ={ 1:'one', [2,10]:'two and ten',11:'eleven'}
print(numb)

Output:

TypeError: unhashable type: 'list'

Here above, we have considered a straightforward example. Here we have used a dictionary to create a number dictionary and then tried to print it. But instead of the output, we get an error because we have used a list type in it. In the next section, we will see how to eliminate the error.

But Before that let us also look at an another example.

country=[
    {
    "name":"India",[28,7]:"states and area",
    "name":"France",[27,48]:"states and area"}
]
print(country)

TypeError: unhashable type: 'list'

Here in the above example, we can see we come across the same problem. Here in this dictionary, we have taken the number of states and their ranking worldwide as the data. Now let’s quickly jump to the next section and eliminate these errors.

Troubleshooting:”unhashable type:list”

In this section, we will try to get rid of the errors. Let us start with the first example here. To rectify it all, we have to do is use a tuple.

numb ={ 1:'one', tuple([2,10]):'two and ten',11:'eleven'}
print(numb)

{1: 'one', (2, 10): 'two and ten', 11: 'eleven'}

With just a slight code change, we can rectify this. Here we have used a tuple, which is a hashable data data-type. Similarly, we can rectify the second error of the second example.

country=[
    {
    "name":"India",tuple([28,7]):"states and area",
    "name":"France",tuple([27,48]):"states and area"}
]
print(country)

[{'name': 'France', (28, 7): 'states and area', (27, 48): 'states and area'}]

Again with the help of tuple we are able to rectify it. It is a simple error and can be rectified easily.

Difference between hashable and unhashable type

In this section, we see the basic difference between the 2 types. Also, we classify the various data-types that we use while coding in python under these 2 types.

CONCLUSION

In this article, we covered unhashable type: list error. We looked at why it occurs and the methods by which we can rectify it. To achieve it, we looked at a couple of examples. In the end, we can conclude that this error arises when we use an unhashable data-type in a dictionary.

I hope this article was able to clear all doubts. But in case you have any unsolved queries feel free to write them below in the comment section. Done reading this why not read about the cPickle module.

The post How to Solve “unhashable type: list” Error in Python appeared first on Python Pool.

 At the end of Day 2 of the contest, still only one entry. If this keeps up, by next Monday there will not be a draw for a prize, and we will have a winner by default.

The submission was based on the use of __slots__. In playing around with similar cases, I found an AttributeError message that I had not seen before.  Here's a sample code.

class F:
    __slots__ = ["a"]
    b = 1

f = F()
f.b = 2

What happens if I execute this code using Friendly-traceback. Normally, there would be an explanation provided below the list of variables. Here we see nothing.

Let's inspect by using the friendly console.

I'll have to take care of this later today. Perhaps you know of other error messages specific to the use of __slots__. If so, and if you are quick enough, you could enter the contest. ;-)

import random

user_action = input("Enter a choice (rock, paper, scissors): ")

possible_actions = ["rock", "paper", "scissors"]
computer_action = random.choice(possible_actions)

print(f"\nYou chose {user_action}, computer chose {computer_action}.\n")

Read the full article at https://realpython.com/python-rock-paper-scissors/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

This article is an excerpt from the broader set of changes to our documentation in preparation for Zato.

High-level overview

Zato is a highly scalable, Python-based integration platform for APIs, SOA and microservices. It is used to connect distributed systems or data sources and to build API-first, backend applications. The platform is designed and built specifically with Python users in mind.

Zato is used for enterprise, business integrations, data science, IoT and other scenarios that require integrations of multiple systems.

Real-world, production Zato environments include:

• A platform for processing payments from consumer devices

• A system for a telecommunication operator integrating CRM, ERP, Billing and other systems as well as applications of the operator's external partners

• A data science system for processing of information related to securities transactions (FIX)

• A platform for public administration systems, helping achieve healthcare data interoperability through the integration of independent data sources, databases and health information exchanges (HIE)

• A global IoT platform integrating medical devices

• A platform to process events produced by early warning systems

• Backend e-commerce systems managing multiple suppliers, marketplaces and process flows

• B2B platforms to accept and process multi-channel orders in cooperation with backend ERP and CRM systems

• Platforms integrating real-estate applications, collecting data from independent data sources to present unified APIs to internal and external applications

• A system for the management of hardware resources of an enterprise cloud provider

• Online auction sites

• E-learning platforms

Zato offers connectors to all the popular technologies, such as REST, SOAP, AMQP, IBM MQ, SQL, Odoo, SAP, HL7, Redis, MongoDB, WebSockets, S3 and many more.

Running on premises, in the cloud, or under Docker, Kubernetes and other container technologies, Zato services are optimised for high performance - it is easily possible to run hundreds and thousands of services on typical server instances as offered by Amazon, Google Cloud, Azure or other cloud providers.

Zato servers offer high availability and no-downtime deployment. Servers form clusters that are used to scale systems both horizontally and vertically.

The software is 100% Open Source with commercial and community support available

A platform and language for interesting, reusable and atomic services

Zato promotes the design of, and helps you build, solutions composed of services which are interesting, reusable and atomic (IRA):

• I for Interesting - each service should make its clients want to use it more and more. People should immediately see the value of using the service in their processes. An interesting service is one that strikes everyone as immediately useful in wider contexts, preferably with few or no conditions, prerequisites and obligations. An interesting service is aesthetically pleasing, both in terms of its technical usage as well as in its potential applicability in fields broader than originally envisaged. If people check the service and say "I know, we will definitely use it" or "Why don't we use it" you know that the service is interesting. If they say "Oh no, not this one again" or "No, thanks, but no" then it is the opposite.
• R for Reusable - services can be used in different, independent business processes
• A for Atomic - each service fullfils a single, atomic business need

Each service is deployed independently and, as a whole, they constitute an implementation of business processes taking place in your company or organisation.

With Zato, developers use Python to focus on the business logic exclusively and the platform takes care of scalability, availability, communication protocols, messaging, security or routing. This lets developers concentrate only on what is the very core of systems integrations - making sure their services are IRA.

Python is the perfect choice for API integrations, SOA and microservices, because it hits the sweet spot under several key headings:

• It is a very high level language, with syntax close to how grammar of various spoken languages works, which makes it easy to translate business requirements into implementation
• Yet, it is a solid, mainstream and full-featured, real programming language rather than a domain-specific one which means that it offers to developers a great degree of flexibility and choice in expressing their needs
• Many Python developers have a strong web programming / open source background which means that it is little effort to take a step further, towards API integrations and backend servers. In turn, this means that it is easy to find good people for API projects.
• Many Python developers have knowledge of multiple programming languages - this is very useful in the context of integration projects where one is typically faced with dozens of technologies, vendors or integration methods and techniques
• Lower maintenance costs - thanks to the language's unique design, Python programmers tend to produce code that is easy to read and understand. From the perspective of multi-year maintenance, reading and analysing code, rather than writing it, is what most programmers do most of the time so it makes sense to use a language which makes it easy to carry out the most common tasks.

In short, Python can be construed as executable pseudo-code with many of its users already having roots in modern server-side programming so Zato, both from a technical and strategic perspective, is a natural choice for complex and sophisticated API solutions as a platform built in the language and designed for Python developers from day one.

More than services

Systems integrations commonly require two more features that Zato offers as well:

• File transfer - allows you to move batch data between locations and to distribute it among systems and APIs

• Single Sign-On (SSO) - a convenient REST interface lets you easily provide authentication and authorisation to users across multiple systems

Next steps

• Start the tutorial to learn more technical details about Zato, including its architecture, installation and usage. After completing it, you will have a multi-protocol service representing a sample scenario often seen in banking systems with several applications cooperating to provide a single, consistent API to its callers.

• Visit the support page if you would like to discuss anything about Zato with its creators

Hello coders!! In this article, we will learn how to make a circle using matplotlib in Python. A circle is a figure of round shape with no corners. There are various ways in which one can plot a circle in matplotlib. Let us discuss them in detail.

Method 1: matplotlib.patches.Circle():

• SYNTAX:
  • class matplotlib.patches.Circle(xy, radius=r, **kwargs)
• PARAMETERS:
  • xy: (x,y) center of the circle
  • r: radius of the circle
• RESULT: a circle of radius r with center at (x,y)

import matplotlib.pyplot as plt 

figure, axes = plt.subplots() 
cc = plt.Circle(( 0.5 , 0.5 ), 0.4 ) 

axes.set_aspect( 1 ) 
axes.add_artist( cc ) 
plt.title( 'Colored Circle' ) 
plt.show()

Output & Explanation:

Output

Here, we have used the circle() method of the matplotlib module to draw the circle. We adjusted the ratio of y unit to x unit using the set_aspect() method. We set the radius of the circle as 0.4 and made the coordinate (0.5,0.5) as the center of the circle.

Method 2: Using the equation of circle:

The equation of circle is:

• x = r cos θ
• y = r sin θ

r: radius of the circle

This equation can be used to draw the circle using matplotlib.

import numpy as np 
import matplotlib.pyplot as plt 

angle = np.linspace( 0 , 2 * np.pi , 150 ) 

radius = 0.4

x = radius * np.cos( angle ) 
y = radius * np.sin( angle ) 

figure, axes = plt.subplots( 1 ) 

axes.plot( x, y ) 
axes.set_aspect( 1 ) 

plt.title( 'Parametric Equation Circle' ) 
plt.show() 

Output & Explanation:

Output

In this example, we used the parametric equation of the circle to plot the figure using matplotlib. For this example, we took the radius of the circle as 0.4 and set the aspect ratio as 1.

Method 3: Scatter Plot to plot a circle:

A scatter plot is a graphical representation that makes use of dots to represent values of the two numeric values.  Each dot on the xy axis indicates value for an individual data point.

• SYNTAX:
  • matplotlib.pyplot.scatter(x_axis_data, y_axis_data, s=None, c=None, marker=None, cmap=None, vmin=None, vmax=None, alpha=None, linewidths=None, edgecolors=None)
• PARAMETERS:
  • x_axis_data-  x-axis data
  • y_axis_data- y-axis data
  • s- marker size
  • c- color of sequence of colors for markers
  • marker- marker style
  • cmap- cmap name
  • linewidths- width of marker border
  • edgecolor- marker border-color
  • alpha- blending value

import matplotlib.pyplot as plt 

plt.scatter( 0 , 0 , s = 7000 ) 

plt.xlim( -0.85 , 0.85 ) 
plt.ylim( -0.95 , 0.95 ) 

plt.title( "Scatter plot of points Circle" ) 
plt.show()

Output & Explanation:

Output

Here, we have used the scatter plot to draw the circle. The xlim() and the ylim() methods are used to set the x limits and the y limits of the axes respectively. We’ve set the marker size as 7000 and got the circle as the output.

Method 4: Matplotlib hollow circle:

import matplotlib.pyplot as plt 

plt.scatter( 0 , 0 , s=10000 ,  facecolors='none', edgecolors='blue' ) 

plt.xlim( -0.5 , 0.5 ) 
plt.ylim( -0.5 , 0.5 ) 

plt.show()

Output & Explanation:

Output

To make the circle hollow, we have set the facecolor parameter as none, so that the circle is hollow. To differentiate the circle from the plane we have set the edgecolor as blue for better visualization.

Method 5: Matplotlib draw circle on image:

import matplotlib.pyplot as plt
import matplotlib.patches as patches
import matplotlib.cbook as cb


with cb.get_sample_data('C:\\Users\\Prachee\\Desktop\\cds\\img1.jpg') as image_file:
    image = plt.imread(image_file)

fig, ax = plt.subplots()
im = ax.imshow(image)
patch = patches.Circle((100, 100), radius=80, transform=ax.transData)
im.set_clip_path(patch)

ax.axis('off')
plt.show()

Output & Explanation:

Output

In this example, we first loaded our data and then used the axes.imshow() method. This method is used to display data as an image. We then set the radius and the center of the circle. Then using the set_clip_path() method we set the artist’s clip-path.

Method 6: Matplotlib transparent circle:

import matplotlib.pyplot as plt 

figure, axes = plt.subplots() 
cc = plt.Circle(( 0.5 , 0.5 ), 0.4 , alpha=0.1) 

axes.set_aspect( 1 ) 
axes.add_artist( cc ) 
plt.title( 'Colored Circle' ) 
plt.show()

Output & Explanation:

Output

To make the circle transparent we changed the value of the alpha parameter which is used to control the transparency of our figure.

Conclusion:

With this, we come to an end with this article. These are the various ways in which one can plot a circle using matplotlib in Python.

However, if you have any doubts or questions, do let me know in the comment section below. I will try to help you as soon as possible.

Happy Pythoning!

The post 6 Ways to Plot a Circle in Matplotlib appeared first on Python Pool.

This article covers the most popular GUI Frameworks in Python.

One of Python’s strongest selling points is the vast number of GUI libraries available for GUI development. GUI development can be a tricky task, but thanks to the tools these Python GUI frameworks provide us, things become much simpler.

While some of the below GUI libraries are similar and directly compete with each other, each library has it’s own pros and cons. Sometimes you have special libraries designed for a specific situation, like Kivy is for touchscreen devices. So you don’t have to learn just one.

There are a large number of GUI frameworks in Python and we couldn’t possibly cover all of them. Hence we’ll just be discussing 5 of the most popular and important GUI frameworks in Python.

Tkinter GUI

I decided to start with Tkinter as it’s probably the oldest and most well known GUI framework in Python.

Tkinter was released in 1991 and quickly gained popularity due to its simplicity and ease of use compared to other GUI toolkits at the time. In fact, Tkinter is now included in the standard Python Library, meaning you don’t have to download and install it separately.

Other plus points include the fact that Tkinter has a pretty small memory footprint and a quick start up time. If you were to convert a Tkinter application into an exe with something like pyinstaller, it’s size would smaller than the other GUI library equivalents.

The only downsides to Tkinter are it’s rather outdated and old design. If you’re goal is to create a sleek and modern-looking GUI, Tkinter probably isn’t the best choice. Another possible downside is that Tkinter has fewer “special” widgets than the others, such as a VideoPlayer widget. Such widgets are used rarely, but still important.

You can begin learning it with our very own Tkinter Tutorial series.

PyQt5

PyQt5 is the Python binding of the popular Qt GUI framework which is written in C++.

PyQt5’s main plus points is it’s cross platform ability and modern looking GUI. Personally I’ve noticed quite a few people switching from Tkinter to PyQt5 to be able to create for stylish GUI’s.

Another one of PyQt5’s plus points is the Qt Designer. The Qt Designer is a drag and drop kind of tool where you don’t have to code in each widget individually. Instead, you can simply “drag” the widget and “drop” it onto the screen to create a GUI. It’s similar to Windows Form (VB.NET) and the Scene Builder (JavaFX).

PyQt5 downsides include it’s relatively large package size and slow start up speed. Furthermore, PyQt was released under the GPL License. This means you cannot distribute any software containing PyQt code without bundling the source code with it as well. For someone selling commercial software, this a significant set back. You’ll have to buy a special commercial license which gives you the right to withhold the source code.

The license issue isn’t something just should bother the average programmer though. You can begin learning PyQt from our very own tutorial series here!

If you’ve narrowed down your GUI of choice between Tkinter and PyQt5 and are having a hard time picking one, I suggest you read this comparison article that compares both in a very detailed manner.

PySide2

We’re bringing up PySide right after PyQt5 due to their strong connection. PySide is also a Python binding of the popular Qt GUI framework. Because of this reason the syntax is almost the exact same with some very minor differences.

The reason why PyQt is used more nowadays is because it’s development was faster than that of PySide. When Qt5 was released, PyQt released their binding for it (called PyQt5) in 2016. Whereas it took PySide an extra 2 years to release PySide2 in 2018. If both had released at the same time, things might have been a bit different today.

All the plus points for PyQt5 also apply for PySide2, with an extra addition. Unlike PyQt5, PySide was released under the LGPL license, allowing you to keep the source code for your distributed programs private. This makes the selling of commercial applications easier than it would be using PyQt5.

You can learn more about PyQt5 vs PySide2 from this article here.

Kivy

Kivy is an opensource multi-platform GUI development library for Python and can run on iOS, Android, Windows, OS X, and Linux.

The Kivy framework is well known for it’s support for touchscreen devices and it’s clean and modern looking GUI’s. It’s GUI and widgets have the interactive, multi-touch kind of ability that’s required for any decent GUI on a touchscreen device like a mobile.

The one possible downside to GUI’s created with Kivy is the non-native look. This may or may not be something you wish to have. Other issues may include the smaller community and lack of documentation compared to more popular GUI libraries like Tkinter.

If you’re looking to be developing on Desktop mostly, then it’s better to stick to one of the Qt options. Mobile support is Kivy’s greatest draw after all.

wxPython

wxPython is a Python open source cross platform GUI toolkit. Similar to how PyQt5 is based of the Qt GUI framework, WxPython is also based of a GUI framework called wxWidgets written in C++.

It’s purpose is to allow Python developers to create native user interfaces for their GUI applications on a wide variety of different operating systems.

The native GUI ability makes GUI’s created by wxPython looks very natural on any Operating system that they are run. Although some people may not want to have this native GUI look, instead preferring to have one look/style that is the exact same across all platforms.

This marks the end of the Python GUI Frameworks article. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the article content can be asked in the comments section below.

The post Python GUI Frameworks appeared first on CodersLegacy.

This week we welcome Claudia Regio (@ClaudiaRegio) as our PyDev of the Week! Claudia is a program manager for Python Data Science with a focus on Python Notebooks in Visual Studio Code at Microsoft. She also blogs on Microsoft’s dev blog.

Let’s spend some time getting to know Claudia better!

Can you tell us a little about yourself (hobbies, education, etc):

I am originally from Italy and moved to the Greater Seattle Area when I was 4 years old. Growing up I lived and breathed squash, and had it not been for COVID I still would be! I have been and always will be a huge math nerd and have been tutoring in math for over 10 years now. I attended the University of Washington where I majored in Applied Physics and received two minors in Comprehensive Mathematics and Applied Mathematics while a member of both the Delta Zeta Sorority and the UW Men’s Squash Team.

After graduating I pursued a Data Science Certificate from the University of Washington to enhance my data analysis + data science skills while working at T-Mobile as a Systems Network Architecture Engineer.

Two years after working in that role, I transitioned to Program Manager at Microsoft for the Python Extension in VS Code, focusing on the development of the Data Science & AI components and features.

Why did you start using Python?

The courses in my data science certificate got me started on Python back in 2017.

What other programming languages do you know and which is your favorite?

I learned Java during my time in college and while I enjoyed Java being a strongly typed language, no language beats Python when it comes to data science.

What projects are you working on now?

I am currently managing the Python + Jupyter Extension partnership in VS Code. While our recently released Jupyter Extension provides Jupyter Notebook support for other languages in VS Code Insiders, I focus on the collaboration of the two extensions to create an optimal notebooks experience for data scientists using Python.

Which Python libraries are your favorite (core or 3rd party)?

Scikit-learn will forever have my heart 3 What do you see as the best features of Python Notebooks?

Our best features in Python Notebooks currently include the variable explorer, data viewer, and my personal favorite, Gather (a complimentary VS Code extension).

When experimenting and prototyping in a notebook, it can often become busy as a user explores different approaches. After eventually reaching the desired result (for instance a specific visualization of the data) a user would then need to manually curate the cells involved with this specific flow. This task can be laborious and error-prone, leaving users without a strong approach for aggregating related code. A second scenario that is common among Notebooks users is that software engineers are tasked with turning an existing notebook into a production-ready script. The process of pulling out unneeded imports, graphs, outputs is often highly time consuming and can lead to errors as well. Gather is a complimentary VS Code extension that grabs all the relevant and dependent code required to create the contents of a selected cell and extracts those code dependencies into a new notebook or Python script. This helps save data scientists and developers a lot of notebook cleaning time!

Why should Python developers and data scientists use Visual Studio Code over another editor?

VS Code is a free and open source editor with a family of extensions (from both Microsoft and the open source community), products, and features that aim to make a seamless experience for developers and data scientists. A few examples include:

• Python (Comes with the Jupyter Extension): Includes features such as IntelliSense, linting, debugging, code navigation, code formatting, Jupyter notebook support, refactoring, variable explorer, test explorer, snippets, and more!
• Pylance: Language server that supercharges your Python IntelliSense experience with rich type information, helping you write better code faster.
• Live Share: Enables you to collaboratively edit and debug with others in real-time, regardless of what programming languages you’re using or app types you’re building. It allows you to instantly (and securely) share your current project, and then as needed, share debugging sessions, terminal instances, localhost webapps, voice calls, and more!
• Gather: A code cleaning tool that uses a static analysis technique to find and then copy all of the dependent code that was used to generate that cell’s result into a new notebook or script.
• Coding Pack for Python Installer: An installer pack that helps students and new coders quickly get started by installing VS Code, all of the extensions above, as well as Python and common packages such as numpy and pandas.
• Azure Machine Learning: Easily build, train, and deploy machine learning models to the cloud or the edge with Azure Machine Learning service from the Visual Studio Code interface.
• Over 350+ community contributed Python-related extensions on the VS Code Marketplace!

It is the partnership constructed amongst these extensions and the open-source community as well as the Developer Division mindset to always build for the customer that creates an unmatchable experience for both developers and data scientists in VS Code.

Is there anything else you’d like to say?

I would like to thank the incredible team I get to work with (David Kutugata, Don Jayamanne, Ian Huff, Jim Griesmer, Joyce Er, Rich Chiodo, Rong Lu) who make this tool come to life and a thank you to all the customers who engage with us and are helping us build the best tool for data scientists!

If anyone would like to provide any additional feedback, feature requests, or help contribute back to the product you can do so here!

Thanks for doing the interview, Claudia!

The post PyDev of the Week: Claudia Regio appeared first on Mouse Vs Python.

This tutorial details how to set up social auth with Django and Django Allauth.

Introduction

It’s been quite some time since we wrote on any “engineering-like” topic. As we all want to stay efficient and productive, it is a good time to revisit Google Colaboratory.

Google Colaboratory, or Colab for short, has been a great platform for data scientists or machine-learning enthusiasts in general. It offers a free instance of GPU and TPU for a limited time plus it serves a prettifiied version of a Jupyter notebook. It is a great combination of variouofmaller or mid-size projects.

Unfortunately, it comes with certain limitations. The biggest ones are the lack of storage persistency, as well as being sort of confined to a single document. Both limitations complicate the development and make working with multiple files less straightforward.

While some good solutions have been developed by the community (including my previous work here and here), many of us are still on the lookout for something like a “data studio” aka Jupyter Lab.

In this article, we will show how to install and run a Jupyter Lab instance on the Google machine through Colab, turning it into a custom solution with Jupyter Lab frontend and GPU/TPU backend for free. What is more, the approach presented here is generic and will allow you to other services such as Flask as well. It differs from solutions presented here or here, as they show how to connect the Jupyter Colab frontend to a local instance. Here, we will do the exact opposite, so stay on!

General idea

The main idea is to utilize the server that resides behind the Colab notebook, and uses its backend powers, but replacing the frontend. For it to work, the steps go as follows:

• Tap to the server behind the notebook.
• Install all the packages we need (e.g. Jupyter Lab).
• Establish a communication channel.
• Connect to it and have fun.

Getting started

Go over to https://colab.research.google.com to start a new instance, connect to it, and wait for the resources to be allocated. If you want, now is the time to switch the backend to either GPU or TPU (unless you want to repeat all the steps).

✕

Figure 1.

The proof we have connected to the Google backend.

Preparing the workspace

The first “hack”

Now, we need to go deeper and talk to the machine behind the notebook rather than with the notebook itself. The standard way to interact with the shell underneath is to prefix bash commands with ! (e.g. !ls -la). However, it may generate some problems later, so it is better to use an alternative way, mainly execute

1
eval "$SHELL"

in a cell, which will let us communicate directly with the console behind.

Installing Jupyter Lab

Next, we install Jupyter Lab or any other thing for that matter. Natively, Colab does not have it installed, which you can confirm by executing:

1
2
3
4
5
6
7
8
9
!pip list | egrep jupyter

# output
jupyter                       1.0.0          
jupyter-client                5.3.5          
jupyter-console               5.2.0          
jupyter-core                  4.7.0          
jupyterlab-pygments           0.1.2          
jupyterlab-widgets            1.0.0  

1
2
3
4
5
6
7
8
9
10
11
12
13
!pip install jupyterlab
!pip list | egrep jupyter

# output
jupyter                       1.0.0          
jupyter-client                6.1.11         
jupyter-console               5.2.0          
jupyter-core                  4.7.0          
jupyter-server                1.2.2          
jupyterlab                    3.0.5          
jupyterlab-pygments           0.1.2          
jupyterlab-server             2.1.2          
jupyterlab-widgets            1.0.0          

So now, we have all we need when it comes to the Python environment, but we still need to expose it outside of the notebook. For this, we will do the so-called reverse ssh tunnel.

Reverse SSH tunnel

The reverse SSH tunneling allows using the existing connection between two machines to set up a new connection channel back from the local machine to the remote one. As this article explains:

Because the original connection came from the remote computer to you, using it to go in the other direction is using it “in reverse.” And because SSH is secure, you’re putting a secure connection inside an existing secure connection. This means your connection to the remote computer acts as a private tunnel inside the original connection.

Now, as per the vocabulary used by the article, the local machine is actually the Google server that runs Colab. It is this machine’s port we would like to expose to the outside world. However, as we don’t know the outside address of our local machine (or the “remote” one as per the article’s vocabulary), we use a third-party service, namely http://localhost.run/.

This solution was suggested by haqpl, who is a professional pentester and a friend of mine.

It acts as both the end to the reverse SSH tunnel and a normal HTTP server, allowing us to use it as a bridge in communication. In other words, the service completes the SSH tunnel on one end, and an HTTP server on the other connecting our local PC to whatever service we run on Colab.

Generate a public key

Before we start, there is one thing we n ed to take care of. We need a key pair to secure the SSH channel without a password.

This is the easy part. Detailed instructions can be found on GitHub. For us, it is enough we execute the following commands. Don’t worry about a passphrase. Just hit enter.

1
ssh-keygen -t ed25519 -C "your_email@example.com"

By default, the keys are stored under /root/.ssh/id_ed25519.pub. Next, confirm we have the ssh-agent and register the key.

1
2
3
4
5
eval "$(ssh-agent -s)"
ssh-add

# expected response
Identity added: /root/.ssh/id_ed25519 (your_email@example.com)

At this point, we are ready to test the tunnel.

Test the connection

To initialize the connection, we need to pick a port that is unlikely to be used by the system already. For example 9999. It’s a nice number, isn’t it? Then, the command to execute will map this port to port 80 (the standard port for HTTP connection). Additionally, we need to make the system turn a blind eye to who the host is. Hence the -o flag.

1
ssh -o StrictHostKeyChecking=no -R 80:localhost:9999 ssh.localhost.run

If all goes well, the last line of the response should give you the URL of where to point your local machine to.

✕

Figure 2.

The SSH reverse tunnel has been established. In our case, the URL is `root-3e42408d.localhost.run`.

However, when you copy-paste it to your browser, the most likely response you will get is Something went wrong opening the port forward, check your SSH command output for clues!. This is OK, as there is really no service running at this port (yet).

Let’s start a small python server under this port (or change it if you used it before).

1
python -m http.server 9999 & ssh -o StrictHostKeyChecking=no -R 80:localhost:9999 ssh.localhost.run

When the connection is established, you should be able to browse through the files on Colab in your browser, seeing lines like this:

1
2
3
4
5
===============================================================================
root-be893e68.localhost.run tunneled with tls termination
127.0.0.1 - - [14/Jan/2021 22:39:21] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [14/Jan/2021 22:39:22] code 404, message File not found
127.0.0.1 - - [14/Jan/2021 22:39:22] "GET /favicon.ico HTTP/1.1" 404 -

printed out in the notebook.

This is a very good sign! It means that if we can run python -m http.server, we can just as well replace it with Flask or Jupyter Lab, and this is exactly what we are about to do!

The final hit

Now, let’s change the port and start the Lab instead. The command to run is a bit lengthy, and the reason is that we must pass the necessary arguments:

• --ip=0.0.0.0 to stress this is a local machine (local to Colab).
• --port=8989, of course,
• --allow-root, otherwise Lab will start but refuse to communicate with you.

1
jupyter lab --ip=0.0.0.0 --port=8989 --allow-root & ssh -o StrictHostKeyChecking=no -R 80:localhost:8989 ssh.localhost.run

Again, if all goes correctly, then by virtue of the SSH tunnel, we should access the Jupyter Lab externally using the URL issued earlier. The only add-ons are will be the parameters and the token you can read from the response.

✕

Figure 3.

The confirmation that the Jupyter Lab instance is running.

Take the URL given and copy-paste it to another tab in your browser. Remember to replace the localhost:8989 with the URL received earlier.

✕

Figure 4.

The confirmation that the Jupyter Lab is indeed connected to the Colab backend.

Conclusion

This is it! The combination of Jupyter Lab and Google Colaboratory that we created thanks to reverse SSH tunneling (and haqpl), gives probably the ultimate freedom, as now you have simplified access to upload/download of your files, convenience of organizing your project across multiple files, and support of really powerful hardware to your calculations… for free.

One word, before we go… Remember that despite the SSH channel is secure, the session is open to whoever knows of your URL. If, as a part of your work, you decided to attach e.g. Google Drive to the machine, there is a chance someone may access your files even without you knowing. So please, use this “hack” carefully. Alternatively, you may consider using a virtual private server (VPS) to replace the localhost.run and give exclusive ownership of the endpoint.

Thanks for reading! Please, let me know in the comments in case you stumble across problems or have any suggestions. Good luck and have fun!

Flourish is a visual toy app that draws harmonographs, sinuous curves simulating a multi-pendulum trace:

Each harmonograph is determined by a few dozen parameter values, which are chosen randomly. The number of parameters depends on the number of pendulums, which defaults to 3.

Click a thumbnail to see a larger version. The large-image pages have thumbnails of “adjacent” images. Each harmonograph is determined by a few dozen parameter values. For each parameter, four nearby values are substituted, giving four thumbnails for each parameter. Clicking an adjacent thumbnail continues your exploration of the parameter space:

The settings dialog lets you adjust the number of pendulums (which determines the number of parameters) and the kinds of symmetry you are interested in.

I started this because I wanted to understand how the parameters affected the outcome, but I was also interested to give it a purely visual design. As an engineer, it was tempting to present the values of the parameters quantitatively, but I like the simplicity of just clicking curves you like.

I repeated a trick I’ve used in other mathy visual toys: when you download a PNG file of an image, the parameter values are stored in a data island in the file. You can re-upload the image, and Flourish will extract the parameters and put you back into the parameter-space exploration at that point.

This is one of those side projects that let me use different sorts of things than I usually do: numpy, SVG, sass, Docker, and so on. I had more ideas for things to add (there is color in the code but not the UI). Maybe someday I will build them.

BTW, I am happy that my first post of 2021 is called “Flourish.” I hope it is a harbinger of things to come.

Hello coders!! In this article, we will be learning how one can capitalize the first letter in the string in Python. There are different ways to do this, and we will be discussing them in detail. Let us begin!

Method 1: str.capitalize() to capitalize the first letter of a string in python:

• Syntax: string.capitalize()
• Parameters: no parameters
• Return Value: string with the first capital first letter

string = "python pool"
print("Original string:")
print(string)
print("After capitalizing first letter:")
print(string.capitalize())

Output & Explanation:

Output

When we use the capitalize() function, we convert the first letter of the string to uppercase. In this example, the string we took was “python pool.” The function capitalizes the first letter, giving the above result.

Method 2: string slicing + upper():

• Synatx: string.upper()
• Parameters: No parameters
• Return Value:  string where all characters are in upper case

string = "python pool"
print("Original string:")
print(string) 
result = string[0].upper() + string 
print("After capitalizing first letter:")
print(result) 

Output & Explanation:

Output

We used the slicing technique to extract the string’s first letter in this example. We then used the upper() method to convert it into uppercase.

Method 3: str.title():

• Syntax: str.title()
• Parameters: a string that needs to be converted
• Return Value: String with every first letter of every word in capital

string = "python pool"
print("Original string:")
print(string)
print("After capitalizing first letter:")
print(str.title(string))

Output & Explanation:

Output

str.title() method capitalizes the first letter of every word and changes the others to lowercase, thus giving the desired output.

Method 4: capitalize() Function to Capitalize the first letter of each word in a string in Python

string = "python pool"
print("Original string:")
print(string)
print("After capitalizing first letter:")
result = ' '.join(elem.capitalize() for elem in string.split())
print(result)

Output & Explanation:

Output

In this example, we used the split() method to split the string into words. We then iterated through it with the help of a generator expression. While iterating, we used the capitalize() method to convert each word’s first letter into uppercase, giving the desired output.

Method 5: string.capwords() to Capitalize first letter of every word in Python:

• Syntax: string.capwords(string)
• Parameters: a string that needs formatting
• Return Value: String with every first letter of each word in capital

import string
txt = "python pool"
print("Original string:")
print(txt)
print("After capitalizing first letter:")
result = string.capwords(txt)
print(result)

Output & Explanation:

Output

capwords() function not just convert the first letter of every word into uppercase. It also converts every other letter to lowercase.

Method 6: Capitalize the first letter of every word in the list in Python:

colors=['red','blue','yellow','pink']
print('Original List:')
print(colors)
colors = [i.title() for i in colors]
print('List after capitalizing each word:')
print(colors)

Output & Explanation:

Output

Iterate through the list and use the title() method to convert the first letter of each word in the list to uppercase.

Method 7:Capitalize first letter of every word in a file in Python

file = open('sample1.txt', 'r') 
for line in file: 
    output = line.title() 
 
    print(output)

Output & Explanation:

Python Pool Is Your Ultimate Destination For Your Python Knowledge

We use the open() method to open the file in read mode. Then we iterate through the file using a loop. After that, we capitalize on every word’s first letter using the title() method.

You might be interested in reading:

• How to Convert String to Lowercase in Python
• How to use Python find() | Python find() String Method
• Python Pass Statement| What Does Pass Do In Python
• How to Count Words in a String in Python

Conclusion:

The various ways to convert the first letter in the string to uppercase are discussed above. All functions have their own application, and the programmer must choose the one which is apt for his/her requirement.

However, if you have any doubts or questions, do let me know in the comment section below. I will try to help you as soon as possible.

Happy Pythoning!

The post 7 Ways in Python to Capitalize First Letter of a String appeared first on Python Pool.

In this episode, we'll be discussing two powerful tools for data reporting and exploration: Datasette and Dogsheep. <br/> <br/> Datasette helps people take data of any shape or size, analyze and explore it, and publish it as an interactive website and accompanying API. <br/> <br/> Dogsheep is a collection of tools for personal analytics using SQLite and Datasette. Imagine a unified search engine for everything personal in your life such as twitter, photos, google docs, todoist, goodreads, and more, all in once place and outside of cloud companies. <br/> <br/> On this episode we talk with Simon Willison who created both of these projects. He's also one of the co-creators of Django and we'll discuss some early Django history!<br/> <br/> <strong>Links from the show</strong><br/> <br/> <div><b>Datasette</b>: <a href="https://datasette.io/" target="_blank" rel="noopener">datasette.io</a><br/> <b>Dogsheep</b>: <a href="https://dogsheep.github.io/" target="_blank" rel="noopener">dogsheep.github.io</a><br/> <b>Datasheet newsletter</b>: <a href="https://datasette.substack.com/" target="_blank" rel="noopener">datasette.substack.com</a><br/> <b>Video: Build your own data warehouse for personal analytics with SQLite and Datasette</b>: <a href="https://www.youtube.com/watch?v=CPQCD3Qxxik" target="_blank" rel="noopener">youtube.com</a><br/> <br/> <b>Examples</b><br/> <b>List</b>: <a href="https://github.com/simonw/datasette/wiki/Datasettes" target="_blank" rel="noopener">github.com</a><br/> <b>Personal data warehouses</b>: <a href="https://simonwillison.net/2020/Nov/14/personal-data-warehouses/" target="_blank" rel="noopener">github.com</a><br/> <b>Global power plants</b>: <a href="https://global-power-plants.datasettes.com/" target="_blank" rel="noopener">datasettes.com</a><br/> <b>SF data</b>: <a href="https://san-francisco.datasettes.com/" target="_blank" rel="noopener">datasettes.com</a><br/> <b>FiveThirtyEight</b>: <a href="https://fivethirtyeight.datasettes.com/" target="_blank" rel="noopener">fivethirtyeight.datasettes.com</a><br/> <b>Lahman’s Baseball Database</b>: <a href="https://baseballdb.lawlesst.net/" target="_blank" rel="noopener">baseballdb.lawlesst.net</a><br/> <b>Live demo of current main</b>: <a href="https://latest.datasette.io/" target="_blank" rel="noopener">datasette.io</a><br/></div><br/> <strong>Sponsors</strong><br/> <br/> <a href='https://talkpython.fm/linode'>Linode</a><br> <a href='https://talkpython.fm/training'>Talk Python Training</a>

tl; dr: Python was wrong ;-)

After one day, I've had one valid entry submitted to the contest I announced yesterday; I've also had two other submissions from the same contributor that I deemed to be invalid for the contest. The submissions shared a similar characteristics to a different degree: the information provided by Python in the exception message did not tell the whole story and, taken on its own, might have been considered to be misleading. 

One such cases which I did not considered to be valid for this contest was the error message UnboundLocalError: local variable 'x' referenced before assignment  given for code containing the following

def f():
    x = 1
    del x
    print(x)  # raises the exception

When the exception is raised, there is indeed no trace of variable "x". So while there was an assignment for such a variable before, after deletion it no longer exists. Reading this code, instead of an UnboundLocalError, the exception that should probably be raised at this point is NameError: name 'x' is not defined ; however, Friendly-traceback's role is not to second-guess Python but to explain what an exception means and, whenever possible, give more details using the information available when the exception was raised. I considered this case and another one to be beyond the scope of what Friendly-traceback could accomplish.

The entry that I deemed to be valid was based on the following code:

class F:
    __slots__ = ["a"]

f = F()
f.b = 1

The error message given by Python in this case is AttributeError: 'F' object has no attribute 'b'. While technically correct, the problem is not that this object has no such attribute but that it cannot have such an attribute. This information can be easily obtained when the exception is raised and the information provided by Friendly-traceback now includes the following:

The object f has no attribute named b. 
Note that object f uses __slots__ which prevents the creation of new 
attributes. The following are some of its known attributes: a.

Reminder: the contest is open for 8 more days.

These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2021-01-16 22:31:10 GMT

1. Extract line from txt file using python - [10/7]
2. find the most valuable vertex among all reachable vertices - [7/3]
3. Pandas: Add an empty row after every index in a MultiIndex dataframe - [7/3]
4. PyGame window stopped showing up - [6/2]
5. Pass weights to scipy.interpolate.griddata() function - [6/0]
6. Unable to convert output to list - [5/6]
7. Double precision is different in different languages - [5/4]
8. Why is my code not showing any output? I am trying to use while loop to debug the error i was getting before it - [5/3]
9. Generating binary sequences without repetition - [5/3]
10. Python: Combining unique values in list of dicts where keys are the same? - [5/3]

Hello geeks and welcome in this article, we will cover cPickle. Along with that, we will also look at some examples to better understand. We will also see what its application are. But before moving that ahead, let us look at the function’s definition to develop a basic understanding of it.

The cPickle module helps us by implementing an algorithm for turning an arbitrary python object into a series of Bytes. The Pickle module also carries a similar type of program. But the difference between the 2 is that cPickle is much faster and implements the algorithm in C. The only drawback of cPickle over Pickle is that it does not allow the user to subclass from Pickle. In short, we can conclude that cPickle is used with the motive of object serialization. Now in the next section, let us analyze our definition through a bunch of programs.

Application of cPickle

In this section we will see the application cPickle. The first here would be to pickle the data .

– PICKLING THE DATA

import pickle as cPickle
mylist = ['apple', 'bat', 'cat', 'dog']
with open('data.txt', 'wb') as fh:
cPickle.dump(mylist, fh)

Here one thing that I would like to clear is that there is no cPickle module available in the newer python version. So import the pickle module, which will automatically use the cPickle. Now coming back to the explanation of the above example. Here first, we have imported our module. After which, we have declared an array. Next, we have used the command with open and specified our file’s name. Here we have used the “wb” mode instead of “w” as all the operations need to be done using the bytes stream.

If you look at the above 2 images, you can notice that a file named data.txt is added to the left-hand side after successfully running the program. You can also see from the 1st image the outcome for importing the cPickle file. Now we are done with the creation part, but still, we can read nothing as the data is in binary form. Next, we will look at how to extract data from the file.

– Extracting data from pickle

import pickle as cPickle
cPickle_off = open("data.txt", "rb")
file = cPickle.load(cPickle_off)
print(file)

Here from the above example, we can note that we have successfully retrieved our pickle data. To achieve this, we have opened our file “data.txt.” Then we have loaded that and eventually printed the result.

cPickle vs Pickle Comparison

This section will discuss the main difference between the cPickle and Pickle Module. The Pickle module is used to serialize and de-serialize the python object. Like the cPickle module, it converts python objects into a byte stream. Which helps in further storing it in the database. The basic difference between them is that cPickle is much faster than Pickle. This is because the cPickle implements the algorithm part in C.

Import error: no module named cPickle

This section will discuss one common error that many of us face when working with the cPickle module. The common error is “no module named cPickle found.” As I have discussed earlier, to avoid such an error, we should import the Pickle module instead of cPickle. What it does is that it automatically imports the C accelerator. So import Pickle and get rid of such errors.

You might be interested in reading:

• Fibonacci series in Python
• Using Pandas to CSV() with Perfection
• MD5 Hash Function: Implementation in Python

Conclusion

In this article, we looked at cPickle. We covered its definition and looked at its application through an example. We looked at the process of pickling and then retrieving the data from it. In the end, we can conclude that the cPickle module is used for object serialization.

I hope this article was able to clear all doubts. But in case you have any unsolved queries feel free to write them below in the comment section. Done reading this, why not read about the ogrid function next.

The post cPickle in Python Explained With Examples appeared first on Python Pool.

Summary

Get a chance of winning a prize by writing code with ONE error that Friendly-traceback cannot properly analyze, in one of three categories:

• SyntaxError: invalid syntax
• SyntaxError: some message, where some message is not recognized.
• Any case of NameError, AttributeError, ModuleNotFoundError, UnboundLocalError, ImportError, IndexError, KeyError that is not recognized or is given an incorrect explanation by Friendly-traceback.

Submitted issues about bugs for Friendly-traceback itself are also considered for this contest.

Links: Github issue

Friendly-traceback documentation

The prize

There will be one prize given drawn randomly from all eligible submissions. The prize consists of one ebook/pbook of your choice with a maximum value of 50 USD (including delivery for pbook) as long as I can order it and have it delivered to you. Alternatively, a donation for that amount to the open source project of your choice if it can be made using Paypal.

The details

Each valid issue will get one entry for the contest. Valid issues contain code that might be expected to be written by a beginner or advanced beginner. It excludes code that uses type annotations as well as the use of async and await keywords.  The code is expected to contain ONE mistake only and not generate secondary exceptions.

The code can be run either using the friendly-console, or running in a Jupyter environment or from an editor as described in the documentation.

For a given valid submission, a bonus entry will be given if a link can be provided to an actual example from a site (such as StackOverflow, /r/python or /r/learnpython, etc.) where a question had been asked prior to this contest.

Exceptions that are not recognized by Friendly-traceback or for which the explanation (in English or French) is wrong or misleading are considered to be valid issues.

Submissions that are considered to be duplicate of previously submitted cases (because they basically have the same cause) will not be considered.

Honor code

I would ask that you do not read the source of Friendly-traceback with the sole intent of finding ways to write code that is designed to lead it to provide incorrect explanations.

End of contest

The contest will end on Monday January 25, at 8 AM Atlantic Standard Time.

Hello geeks and welcome in this article, we will cover Python Tempfile(). Along with that, for an overall better understanding, we will also look at its syntax and parameter. Then we will see the application of all the theory part through a couple of examples. The Tempfile is a standard library used to create temporary files and directories. These kinds of files come really handy when we don’t wish to store data permanently. If we are working with massive data, these files are created with unique names and stored at a default location, varying from os to os. For instance, in windows, the temp folder resides in profile/AppData/Local/Temp while different for other cases.

Creating a Temporary File

import tempfile 
  
file = tempfile.TemporaryFile() 
print(file) 
print(file.name)

Output:

<_io.BufferedRandom name=3>
3

Here, we can see how to create a temporary file using python tempfile(). At first, we have imported the tempfile module, following which we have defined a variable and used our function to create a tempfile. After which, we have used the print statement 2 times 1st to get our file and 2nd to exactly get ourselves the exact filename. The filename is randomly generated and may vary from user to user.

Creating a Named Temporary File

import tempfile

file = tempfile.NamedTemporaryFile()
print(file)
print(file.name)

Output:

<tempfile._TemporaryFileWrapper object at 0x000002756CC7DC40>
C:\Users\KIIT\AppData\Local\Temp\tmpgnp482wy

Here we have to create a named temporary file. The only difference, which is quite evident, is that instead of, Temporary file, we have used NamedTemporaryfile. A random file name is allotted, but it is clearly visible, unlike the previous case. Another thing that can be verified here is the structure profile/AppData/Local/Temp(as mentioned for windows os). As if now, we have seen how to create a temporary file and a named temporary file.

Creating a Temporary Directory

import tempfile
dir = tempfile.TemporaryDirectory() 
print(dir)

Here above we have created a directory. A directory can be defined as a file system structure that contains the location for all the other computer files. Here we can see that there’s just a minute change in syntax when compared to what we were using for creating a temporary file. Here just instead of TemporaryFile, we have used TemporaryDirectory.

Reading and Writing to a Temporary File

import tempfile 
  
file = tempfile.TemporaryFile() 
file.write(b'WELCOME TO PYTHON PPOOL') 
file.seek(0) 
print(file.read()) 
  
file.close()

Output:

b'WELCOME TO PYTHON PPOOL'

Above we can see how to read and write in the temporary files. Here we have first created a temporary file. Following which we have used the write function which is used to write data in a temporary file. You must be wondering what is ‘b’ doing there. The fact is that the temporary files take input by default so ‘b’ out there converts the string into binary. Next, the seek function is used to called to set the file pointer at the starting of the file. Then we have used a read function that reads the content of the temporary file.

Alternative to Python tempfile()

Python tempfile() is great but in this section, we will look at one of the alternatives to it. mkstemp() is a function that does all things you can do with the tempfile but in addition to that, it provides security. Only the user that created that has created the temp file can add data to it. Further, this file doesn’t automatically get deleted when closed.

import tempfile 
   
sec_file = tempfile.mkstemp() 
print(sec_file)

Output:

(3, '/tmp/tmp87gc2pz0')

Here we can how to create a temporary file using mkstemp(). Not much syntax change can be seen here only; instead of TemporaryFile(), we have used mkstemp(), and the rest of everything is the same.

General FAQ’s regarding python tempfile()

1. How to find the path of python tempfile()?

Ans. To get the path of a python tempfile(), you need to create a named tempfile, which has already been discussed. In that case, you get the exact path of the tempfile, as in our case, we get "C:\Users\KIIT\AppData\Local\Temp\tmpgnp482wy".

2. How to perform cleanup for python tempfile()?

Ans. Python itself deletes the tempfile once they are closed.

3. How to create secure python tempfile()?

Ans. In order to create a secure tempfile() we can use the mkstemp() function. As has been discussed in detail already. We know the tempfile created using this can only be edited by creating it. Your permission is also required for someone else to access it.

4. What is the name for python tempfile() ?

Ans. If you create a temporary file, then it has no name, as discussed above. Whereas when you create a Named tempfile, a random name is allocated to it, visible in its path.

Conclusion

In this article, we covered the Python tempfile(). Besides that, we have also looked at creating a temporary file, temporary directory, how to read and write in a temp file, and looked at an alternative called mkstemp(). Here we can conclude that this function helps us create a Temporary file in python.

I hope this article was able to clear all doubts. But in case you have any unsolved queries feel free to write them below in the comment section. Done reading this, why not read about the argpartition function next.

The post Unboxing the Python Tempfile Module appeared first on Python Pool.