TypeError: 'CommandCursor' object has no attribute '__getitem__'

I am getting this TypeError when trying to access Bottle's rest api through Apache server, but it's working properly with Bottle's WSGI server.

Mongodb sample data:

 "_id" : ObjectId("55c4f21782f2811a08b7ddbb"),
 "TestName" : "TestName1",
 "Results" : [
     {
             "Test" : "abc",
             "Log" : "Log information"
     },
     {
             "Test" : "xyz",
             "Log" : "Log information"
     },
]

I want to fetch only those records/sub document where Results.Test="abc"

My Bottle API code:

@route('/TestSampleApi',method='GET')
def GetTestData():
    #request.json will have data passed in url
    pipe = [
            {"$match":{"Results": {"$elemMatch": {'Test':'abc'}}}}, 
                { "$unwind" :"$Results"},
                { "$match": { "Results.Test":'abc'}},
                { "$group" : { "_id" : "$_id", "Results" : { "$addToSet" : "$Results" } }}
            ]           
            res = collection.aggregate(pipeline=pipe)
            get_record=res['result']   #getting TypeError in this line 

    response.set_header('Content-Type',  'application/json')
    return dumps(get_record, indent=4)

Above code working properly with curl -i GET "http://localhost:8080/TestSampleApi?Test=abc"
But not working with apache: curl -i GET "http://HOST_NAME/TestSampleApi?Test=abc"

via Chebli Mohamed

How to group an array by multiple keys?

I'd like a function that can group a list of dictionaries into sublists of dictionaries depending on an arbitrary set of keys that all dictionaries have in common.

For example, I'd like the following list to be grouped into sublists of dictionaries depending on a certain set of keys

l = [{'name':'b','type':'new','color':'blue','amount':100},{'name':'c','type':'new','color':'red','amount':100},{'name':'d','type':'old','color':'gold','amount':100},{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100},{'name':'g','type':'normal','color':'red','amount':100}]

If I wanted to group by type, the following list would result, which has a sublists where each sublist has the same type:

[[{'name':'b','type':'new','color':'blue','amount':100},{'name':'c','type':'new','color':'red','amount':100}],[{'name':'d','type':'old','color':'gold','amount':100},{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100}],[{'name':'g','type':'normal','color':'red','amount':100}]]

If I wanted to group by type and color, the following would result where the list contains sublists that have the same type and color:

[[{'name':'b','type':'new','color':'blue','amount':100}],[{'name':'c','type':'new','color':'red','amount':100}],[{'name':'d','type':'old','color':'gold','amount':100}],[{'name':'e','type':'old','color':'red','amount':100},
{'name':'f','type':'old','color':'red','amount':100}],[{'name':'g','type':'normal','color':'red','amount':100}]]

I understand the following function can group by one key, but I'd like to group by multiple keys:

 def group_by_key(l,i):

      l = [list(grp) for key, grp in itertools.groupby(sorted(l, key=operator.itemgetter(i)), key=operator.itemgetter(i))]

This is my attempt using the group_by_function above

 def group_by_multiple_keys(l,*keys):
      for key in keys:
          l = group_by_key(l,key)
          l = [item for sublist in l for item in sublist]
      return l 

The issue there is that it ungroups it right after it grouped it by a key. Instead, I'd like to re-group it by another key and still have one list of sublists.

via Chebli Mohamed

Solving a system of Quadratic Equations

I am doing a cryptography program in python...I'll give you the summary.

It consists in reading a random phrase, like: "HELLO". Then it assigns it's respective ASCII equal like:

H=72, E=79.

Then, using Pythagoras' Theorem, it creates two numbers C1 and C2, like this:

C1 = sqrt(A^2 + (A+B)^2);
C2= sqrt(B^2 + (A+B)^2)

where, in this case A = H and B = E. That would be the encryption part, but I am having problems solving the system, which will act as the decryptor.

Now here is the question:

How can I solve this system using python?

C1 = sqrt(A^2 + (A+B)^2); 
C2 = sqrt(B^2 + (A+B)^2);

Of course only C1 and C2 are known.

Do I need a new module? Which one?

via Chebli Mohamed

Increasing INSERT Performance in Django For Many Records of HUGE Data

So I've been trying to solve this for a while now and can't seem to find a way to speed up performance of inserts with Django despite the many suggestions and tips found on StackOverflow and many Google searches.

So basically I need to insert a LOT of data records (~2 million) through Django into my MySQL DB, each record entry being a whopping 180KB. I've scaled my testing down to 2,000 inserts yet still cant get the running time down to a reasonable amount. 2,000 inserts currently takes approximately 120 seconds.

So I've tried ALL of the following (and many combinations of each) to no avail:

• "Classic" Django ORM create model and .save()
• Single transaction (transaction.atomic())
• Bulk_create
• Raw SQL INSERT in for loop
• Raw SQL "executemany" (multiple value inserts in one query)
• Setting SQL attributes like "SET FOREIGN_KEY_CHECKS=0"
• SQL BEGIN ... COMMIT
• Dividing the mass insert into smaller batches

Apologizes if I forgot to list something, but I've just tried so many different things at this point, I can't even keep track ahah.

Would greatly appreciate a little help here in speeding up performance from someone who maybe had to perform a similar task with Django database insertions.

Please let me know if I've left out any necessary information!

via Chebli Mohamed

Pause/restart and volume control buttons in a GUI to control the audio recording by using python

I currently have a program to collect sound signal from a mic and play the wave form at real-time on python.I am thinking to add some buttons in a GUI for it. Something like pause and restart buttons. Users can control the sound recording by mouse click on these buttons to stop or restart real-time wave display. And the volume control buttons to increase or reduce the volume of output.

Here is the code:

import pyaudio
import tkinter as tk
import wave
import threading
import queue

import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.lines as line
import numpy as np
from scipy import fftpack
from scipy import signal
    CHUNK = 1024
    FORMAT = pyaudio.paInt16
    CHANNELS = 1

    RATE = 44100
    RECORD_SECONDS = 5
    WAVE_OUTPUT_FILENAME = "output.wav"
    data =[]
    Recording=False
    FFT_LEN = 128
    frames=[]
    counter=1

    #GUI
    class Application(tk.Frame):
        def __init__(self,master=None):
            tk.Frame.__init__(self,master)
            self.grid()
            self.creatWidgets()

        def creatWidgets(self):
            self.quitButton=tk.Button(self,text='quit',command=root.destroy)
            self.quitButton.grid(column=1,row=3)


    #Matplotlib
    fig = plt.figure()
    rt_ax = plt.subplot(212,xlim=(0,CHUNK), ylim=(-10000,10000))
    fft_ax = plt.subplot(211)
    fft_ax.set_yscale('log')
    fft_ax.set_xlim(0,CHUNK/2 + 1)
    fft_ax.set_ylim(1,100000000)
    rt_ax.set_title("Real Time")
    fft_ax.set_title("FFT Time")
    rt_line = line.Line2D([],[])
    fft_line = line.Line2D([],[])

    rt_data=np.arange(0,CHUNK,1)
    fft_data=np.arange(0,CHUNK/2 + 1,1)
    rt_x_data=np.arange(0,CHUNK,1)
    fft_x_data=np.arange(0,CHUNK/2 + 1,1)

    def plot_init():
        rt_ax.add_line(rt_line)
        fft_ax.add_line(fft_line)
        return fft_line,rt_line,

    def plot_update(i):
        global rt_data
        global fft_data

        rt_line.set_xdata(rt_x_data)
        rt_line.set_ydata(rt_data)

        fft_line.set_xdata(fft_x_data)
        fft_line.set_ydata(fft_data)
        return fft_line,rt_line,


    ani = animation.FuncAnimation(fig, plot_update,
                                  init_func=plot_init, 
                                  frames=1,
                                  interval=30,
                                  blit=True)


    # pyaudio
    p = pyaudio.PyAudio()
    q = queue.Queue()

    def audio_callback(in_data, frame_count, time_info, status):
        global ad_rdy_ev

        q.put(in_data)
        ad_rdy_ev.set()
        if counter <= 0:
            return (None,pyaudio.paComplete)
        else:
            return (None,pyaudio.paContinue)


    stream = p.open(format=FORMAT,
            channels=CHANNELS,
            rate=RATE,
            input=True,
            output=False,
            frames_per_buffer=CHUNK,
            stream_callback=audio_callback)


    if Recording:
        wf = wave.open(WAVE_OUTPUT_FILENAME, 'wb')
        wf.setnchannels(CHANNELS)
        wf.setsampwidth(p.get_sample_size(FORMAT))
        wf.setframerate(RATE)

    print("Start Recording")
    stream.start_stream()

    #processing block

    window = signal.hamming(CHUNK)

    def read_audio_thead(q,stream,frames,ad_rdy_ev):
        global rt_data
        global fft_data

        while stream.is_active():
            ad_rdy_ev.wait(timeout=1000)
            if not q.empty():
                #process audio data here
                data=q.get()
                while not q.empty():
                    q.get()
                rt_data = np.frombuffer(data,np.dtype('<i2'))
                rt_data = rt_data * window
                fft_temp_data=fftpack.fft(rt_data,rt_data.size,overwrite_x=True)
                fft_data=np.abs(fft_temp_data)[0:fft_temp_data.size/2+1]
                if Recording :
                    frames.append(data)
            ad_rdy_ev.clear()

    ad_rdy_ev=threading.Event()

    t=threading.Thread(target=read_audio_thead,args=(q,stream,frames,ad_rdy_ev))

    t.daemon=True
    t.start()

    plt.show()
    root=tk.Tk()
    app=Application(master=root)
    app.master.title("Test")
    app.mainloop()

    stream.stop_stream()
    stream.close()
    p.terminate()

    print("* done recording")
    if Recording:
        wf.writeframes(b''.join(frames))
        wf.close()

via Chebli Mohamed

Django helper application's static files not found in admin pages?

Ce résumé n'est pas disponible. Veuillez cliquer ici pour afficher l'article.

Installing a .whl Python package into a specific directory other than the default

I am trying to install the 64 bit version of ntlk, which comes in a .whl file, in a different directory than the standard python34/Lib folder. I am using windows 10 64 bit and python 3.4 64 bit.

I first tried using the instructions in this question (which worked for other modules):

Install a python package into a different directory using pip?

The command I typed into cmd was:

py -m pip install --install-option="--prefix=$PATH_NAME" nltk-3.0.4-py2.py3-none-any.whl

It promptly gave me the following error:

UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.

It appears that I can't install WHL files using the --install-option. Is there an alternate way I can install the .whl package in a non default directory?

Edit: I marked this as solved because the proposed solution allows me to do what I need in my own use case. However, it doesn't completely answer the question due to inherent limitations in using --root to choose an alternate directory. There is technically no correct solution to this problem, see the answer's comments for details.

via Chebli Mohamed