tag:blogger.com,1999:blog-13099775148241602212024-03-27T16:53:21.992-07:00Ali Raza Bhayani - Developer's notesPakistani Developer --> Electronics Engineer by book, Software Developer/ Technopreneur by passion, Open Source Enthusiast, Problem Hacker, Enabler, Do-Tank, jogger and an avid Reader.Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.comBlogger18125tag:blogger.com,1999:blog-1309977514824160221.post-82133170152608004932018-10-05T06:10:00.000-07:002018-10-05T10:33:37.383-07:00Gmail API and Python - Resolved - SMTPAuthenticationError (534, '5.7.14') using Gmail API and Python<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXCeLH-DYWTttd57nB5W5qON_UAikgXdNYDZMB_iKlo_cZaXQoi_5GASqffqJheJ9ijDkXZM2-I4fopfT7p3DdS8W7FtnAcGylHCjxckmg_FyRTraRgll5YJJBG_eIOLeX-qmNGIeE_2Y/s1600/Gmail+API+with+Python.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="319" data-original-width="808" height="126" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXCeLH-DYWTttd57nB5W5qON_UAikgXdNYDZMB_iKlo_cZaXQoi_5GASqffqJheJ9ijDkXZM2-I4fopfT7p3DdS8W7FtnAcGylHCjxckmg_FyRTraRgll5YJJBG_eIOLeX-qmNGIeE_2Y/s320/Gmail+API+with+Python.png" width="320" /></a></div>
<br />
While working on Gmail API and integrating it in a project using Python, I came across this error "SMTPAuthenticationError (534, '5.7.14')" whose solution was not so obvious. I found many people facing the same problem on different forums so I ended up writing a post for future reference. The exact error was:<br />
<br />
<i>"<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">(534, '5.7.14 <<a class="Xx" data-display="https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbsl\n5.7.14" data-sanitized="https://www.google.com/url?q=https://accounts.google.com/signin/continue?sarp%3D1%26scc%3D1%26plt%3DAKgnsbsl%5Cn5.7.14&sa=D&source=hangouts&ust=1538826474935000&usg=AFQjCNGCRHjjfjHiUUdbjRpNiDhcvpBWwA" dir="ltr" href="https://www.google.com/url?q=https://accounts.google.com/signin/continue?sarp%3D1%26scc%3D1%26plt%3DAKgnsbsl%5Cn5.7.14&sa=D&source=hangouts&ust=1538826474935000&usg=AFQjCNGCRHjjfjHiUUdbjRpNiDhcvpBWwA" rel="nofollow noreferrer noopener" tabindex="-1" target="_blank">https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbsl\n5.7.14</a> ....</span></i><br />
<i><span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">> Please log in
via\n5.7.14 your web browser and then try again.\n5.7.14 Learn more
at\n5.7.14 <a class="Xx" data-display="https://support.google.com/mail/answer/78754" data-sanitized="https://www.google.com/url?q=https://support.google.com/mail/answer/78754&sa=D&source=hangouts&ust=1538826474935000&usg=AFQjCNHFTKts-PorESaDTOggscE7fiXxPA" dir="ltr" href="https://www.google.com/url?q=https://support.google.com/mail/answer/78754&sa=D&source=hangouts&ust=1538826474935000&usg=AFQjCNHFTKts-PorESaDTOggscE7fiXxPA" rel="nofollow noreferrer noopener" tabindex="-1" target="_blank">https://support.google.com/mail/answer/78754</a> g65-v6sm13547202pfg.98 - gsmtp')"</span></i><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><br /></span>
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">You need to check and verify following things:</span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><br /></span>
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><b>STEP 1:</b></span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">Recheck your username and password (I know its obvious but still you might be making some mistake while entering the password)</span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><br /></span>
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><b>STEP 2:</b></span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">Check if you have enabled less secure apps by navigating to:</span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"> </span><span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><a href="https://www.google.com/settings/security/lesssecureapps">https://www.google.com/settings/security/lesssecureapps</a></span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><br /></span>
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><b>STEP 3 (Important):</b></span><br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">Disabling Captcha for your Client App which is accessing the Gmail API. Using your browser, login to your Gmail Account and navigate to the link:</span><span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"> </span><br />
<a href="http://www.google.com/accounts/DisplayUnlockCaptcha"><span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">http://www.google.com/accounts/DisplayUnlockCaptcha</span></a><br />
<br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;">Click the continue button. You will see the following message:</span><br />
<blockquote>
"Account access enabled
Please try signing in to your Google account again from your new device or
application."</blockquote>
<br />
Its been found that google resets this setting of UnlockCaptcha if there are any changes in the Gmail Account Settings and therefore the error SMTPAuthenticationError (534, '5.7.14') occurs. You need to go through STEP 3 again so that your App can access Gmail API and send mail.<br />
<span class="tL8wMe EMoHub" dir="ltr" id=":c1.co" style="text-align: left;"><br /></span></div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com12tag:blogger.com,1999:blog-1309977514824160221.post-87843216489904513982018-01-18T00:33:00.000-08:002018-01-24T21:56:13.121-08:00How To Secure Your Redis Installation on CentOS 6.x with Password Authentication | Configuring a Redis Password for CentOS<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlvmefHvA7BAlAolaKE4cYOyqpvSI6lad00_JllVkfkQcgFIFP3R9sJnB1L_DoFZ5mFmurbE0ZK94NAP0YZ1hfhqTK2ldsqiZt76b_Xhl8pzfj1DA1aCmlnwbmQaUsePlOBL4UDvP_YnY/s1600/redis2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="225" data-original-width="225" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlvmefHvA7BAlAolaKE4cYOyqpvSI6lad00_JllVkfkQcgFIFP3R9sJnB1L_DoFZ5mFmurbE0ZK94NAP0YZ1hfhqTK2ldsqiZt76b_Xhl8pzfj1DA1aCmlnwbmQaUsePlOBL4UDvP_YnY/s200/redis2.png" width="200" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
Using CentOS 6.x, for applying Password Authentication in Redis through requirepass Configuration, we need to edit the redis.conf file.<br />
<br />
Open the configuration file(name of the file may vary in your machine):<br />
<br />
<code>/etc/redis/6379.conf</code><br />
<br />
<div style="text-align: left;">
Find <span style="font-size: x-small;"># requirepass foobared</span> and change it in following way:
<br />
<pre 15="" class="brush: bash" highlight:="">requirepass yourpassword
</pre>
Save your changes and restart redis-server
<br />
<pre 15="" class="brush: bash" highlight:="">service redis_6379 restart
</pre>
<pre 15="" class="brush: bash" highlight:="">redis-cli
</pre>
<pre 15="" class="brush: bash" highlight:="">127.0.0.1:6379>set test "TestEntry"
</pre>
Since we have applied password in Redis Configurtion file "/etc/redis/6379.conf" , we will get NOAUTH error as follows:
<br />
<pre 15="" class="brush: bash" highlight:="">(error) NOAUTH Authentication required.
</pre>
while being in redis-cli, write following command with your password for authentication:
<br />
<pre 15="" class="brush: bash" highlight:="">127.0.0.1:6379> auth yourpassword
OK
</pre>
And then when you will set any value, it will return OK
<br />
<pre 15="" class="brush: bash" highlight:="">127.0.0.1:6379> set test "TestEntry"
OK
</pre>
We can also run redis-cli directly with password authentication by using following command:
<br />
<pre 15="" class="brush: bash" highlight:="">redis-cli -a "yourpassword"
</pre>
We get OK when we run redis-cli in this way and set any value:
<br />
<pre 15="" class="brush: bash" highlight:="">127.0.0.1:6379> set test2 "TestEntry2"
OK
</pre>
</div>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com0tag:blogger.com,1999:blog-1309977514824160221.post-33899162623478198262018-01-16T07:36:00.000-08:002018-10-05T22:09:30.679-07:00Python - HTML to Text for sending SMS - SMS Safe characters - remove \xa0 <div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkUR309i1qJQXyrOBxeK2Px1C2WnyAM0OUNTzNi_ruRtSebNokH9Ps312rBZHv52HL-iein1KO5K64MEzBWTkAIVcV32obQVYD4-ORaYJdPkhtG7-s5eND21WvUn57_6MenJizaQLJzrM/s1600/SMS+Python.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="299" data-original-width="600" height="159" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkUR309i1qJQXyrOBxeK2Px1C2WnyAM0OUNTzNi_ruRtSebNokH9Ps312rBZHv52HL-iein1KO5K64MEzBWTkAIVcV32obQVYD4-ORaYJdPkhtG7-s5eND21WvUn57_6MenJizaQLJzrM/s320/SMS+Python.png" width="320" /></a></div>
<br />
Following code are some of the alternatives for removing special characters from string:<br />
<pre 15="" class="brush: python" highlight:="">from bs4 import BeautifulSoup
raw_html = 'Dear Parent,<br />
<span style="font-size: 1rem;">This is a test message, </span><span style="font-size: 1rem;">kindly ignore it. </span><br />
<span style="font-size: 1rem;">Thanks</span><br />
'
clean_text = BeautifulSoup(raw_html, "lxml").text
print clean_text
#u'Dear Parent,\xa0This is a test message,\xa0kindly ignore it.\xa0Thanks'
</pre>
The above code produces these characters \xa0 in the string. To remove them properly, we can use two ways. The first one is BeautifulSoup's get_text method with strip argument as True
<br />
<pre 15="" class="brush: python" highlight:="">clean_text = BeautifulSoup(raw_html, "lxml").get_text(strip=True)
print clean_text
# Dear Parent,This is a test message,kindly ignore it.Thanks
</pre>
The other option is to use python's library unicodedata
<br />
<pre 15="" class="brush: python" highlight:="">import unicodedata
clean_text = BeautifulSoup(raw_html, "lxml").text
print clean_text
#u'Dear Parent,\xa0This is a test message,\xa0kindly ignore it.\xa0Thanks'
new_str = unicodedata.normalize("NFKD",clean_text)
print new_str
# u'Dear Parent,This is a test message,kindly ignore it.Thanks'
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com102tag:blogger.com,1999:blog-1309977514824160221.post-77351768669835072132017-09-27T23:52:00.000-07:002017-09-28T22:50:20.331-07:00Django Python Web App Development - Python XHTML2PDF Library error FIXED - "ImportError: cannot import name inputstream django" Solved<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxn6FK-VGXKpICjiQ2fD1SymIUskBx_BCyQFmbVib-8Q1c3gBSnIzEkm_ZdSSs5bvtKSWz5i2x2fuVnqyHGs3ETy-GIdUZzTbkWZAGyffffrGWAGMKmolplbGvJbDhuhac6JQjM2RwvJo/s1600/python-django_big.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="cannot import name inputstream, Django, Django alirazabhayani, Django Full Stack Development ali raza bhayani, html5lib, inputstream, Python, Python alirazabhayani, xhtml2pdf" border="0" data-original-height="580" data-original-width="821" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxn6FK-VGXKpICjiQ2fD1SymIUskBx_BCyQFmbVib-8Q1c3gBSnIzEkm_ZdSSs5bvtKSWz5i2x2fuVnqyHGs3ETy-GIdUZzTbkWZAGyffffrGWAGMKmolplbGvJbDhuhac6JQjM2RwvJo/s400/python-django_big.png" title="" width="400" /></a></div>
<br />
While working on a Django Web Application using Python, I recently encountered an error while using a library named xhtml2pdf. This library is a Python library for exporting xhtml to PDF.<br />
<br />
While using it in Django Web framework for developing a hassle free web service to export customized HTML templates into PDF, I encountered some error which I had to dig into the library to find the actual problem and its solution. I noticed that this problem of "cannot import name inputstream django" was faced by many people and therefore writing this post to document it. Please note that this proposed solution is workable for Python 2.7 and hasn't been checked on Python v.3.x .<br />
<br />
When I installed xhtml2pdf library and used it in Django Web framework, I encountered following error:<br />
<pre 15="" class="brush: bash" highlight:=""> importerror: cannot import name inputstream
</pre>
Following trace shows that the error is actually triggered from html5lib which is one of the dependencies of xhtml2pdf:
<br />
<pre 15="" class="brush: bash" highlight:="">File "/usr/home/username/virtualenvs/projectname/lib/python2.7/site-packages/xhtml2pdf/parser.py", line 17, in <module>
from html5lib import treebuilders, inputstream
ImportError: cannot import name inputstream
</module></pre>
Due to broken dependencies of xhtml2pdf and lack of documentation, I had to dig in the module which revealed that this error was coming due to the fact that xhtml2pdf requires a specific version of html5lib to work properly. To solve this issue, I installed html5lib's version <b>1.0b8</b> in the following way:
<br />
<pre 15="" class="brush: bash" highlight:="">$ pip install html5lib==1.0b8
</pre>
If you had previously installed html5lib's different version, you will have to uninstall it first by running following command on your terminal or virtual environment:
<br />
<pre 15="" class="brush: bash" highlight:="">$ pip uninstall html5lib
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com8tag:blogger.com,1999:blog-1309977514824160221.post-72693557808727619162016-10-20T03:39:00.000-07:002016-10-20T03:39:00.322-07:00Adding IP aliases in FreeBSD - Freebsd ifconfig add Remove Alias IPv4 / IPv6 / inet6<div dir="ltr" style="text-align: left;" trbidi="on">
FreeBSD Assign IPv4 address
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet 192.168.10.5 netmask 255.255.255.0
</pre>
FreeBSD ifconfig add alias for IPv4
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet 192.168.10.10/24 add
</pre>
FreeBSD ifconfig remove alias for IPv4
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet 192.168.10.10/24 -alias
</pre>
For IPv6
<br />
FreeBSD Enable IPv6 functionality of the interface:
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet6 -ifdisabled
</pre>
FreeBSD Add the IPv6 address 2001:DB8:DBDB::123/48 to the interface em0:
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet6 2001:db8:bdbd::123 prefixlen 48 alias
</pre>
FreeBSD ifconfig remove/delete Alias for IPv6 inet6
<br />
<pre 15="" class="brush: bash" highlight:="">: # ifconfig em0 inet6 2001:db8:bdbd::123/48 delete
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1tag:blogger.com,1999:blog-1309977514824160221.post-57698533909652374652016-08-22T22:15:00.001-07:002016-08-22T23:08:14.760-07:00PostgreSQL setval Sequence - PostgreSQL manually alter sequence - Reset sequence of setval in PostgreSQL - Ali Raza Bhayani<div dir="ltr" style="text-align: left;" trbidi="on">
<br /></div>
In one of the my PostgreSQL tables, I used a very handy builtin Sequence Manipulation function setval() of PostgreSQL to generate Primary Keys in a sequential and controlled manner. But during a migration, I wanted to identify new records by Primary Key sequence greater than thousand. A very handy and tested recipe to restart the setval() function sequence for such cases is by using RESTART WITH in the following way:
<pre 15="" class="brush: bash" highlight:="">
ALTER SEQUENCE sequence_name RESTART WITH 1000;
</pre>
is equivalent to:
<pre 15="" class="brush: bash" highlight:="">
SELECT setval('sequence_name', 1000, FALSE);
</pre>
Either of the statements may be used to restart the sequence and you can get the next value by:
<pre 15="" class="brush: bash" highlight:="">
nextval('sequence_name')
</pre>
The above recipe can also be used to reset PostgreSQL primary key sequence when it falls out of sync.Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com0tag:blogger.com,1999:blog-1309977514824160221.post-61221114288540793592016-08-22T22:15:00.000-07:002016-08-22T23:04:19.420-07:00PostgreSQL setval Sequence - PostgreSQL manually alter sequence - Reset sequence of setval in PostgreSQL - Ali Raza Bhayani<div dir="ltr" style="text-align: left;" trbidi="on">
<br /></div>
In one of the my PostgreSQL tables, I used a very handy builtin Sequence Manipulation function setval() of PostgreSQL to generate Primary Keys in a sequential and controlled manner. But during a migration, I wanted to identify new records by Primary Key sequence greater than thousand. A very handy and tested recipe to restart the setval() function sequence for such cases is by using RESTART WITH in the following way:
<pre 15="" class="brush: bash" highlight:="">
ALTER SEQUENCE seq RESTART WITH 1000;
</pre>
is equivalent to:
<pre 15="" class="brush: bash" highlight:="">
SELECT setval('sequence_name', 1, FALSE);
</pre>
Either of the above statements may be used to restart the sequence and for getting the next value following statement can be used:
<pre 15="" class="brush: bash" highlight:="">
nextval('sequence_name')
</pre>
The above recipe can also be used to reset PostgreSQL primary key sequence when it falls out of sync.Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com8tag:blogger.com,1999:blog-1309977514824160221.post-38762255563719706002016-06-29T06:17:00.003-07:002016-06-29T21:52:39.860-07:00Install and run Redis on Mac OS - Redis Server Installation and First Run<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
Following is the recipe to install Redis server on Mac OS using homebrew in a very simple manner:<br />
<br />
<pre 15="" class="brush: bash" highlight:=""> $ brew install redis
</pre>
<br />
<br />
After installation, you may edit Redis configuration file by going to the path via Mac OS terminal:<br />
<br /></div>
<pre 15="" class="brush: bash" highlight:="">$ /usr/local/etc/redis.conf
</pre>
<br />
To start Redis Server using configuration file on Mac OS, use following command:
<br />
<pre 15="" class="brush: bash" highlight:="">$ redis-server /usr/local/etc/redis.conf
</pre>
<br />
After running the Redis Server on Mac OS, you can check whether Redis Server is running by opening a new terminal window and enter following command:
<br />
<pre 15="" class="brush: bash" highlight:="">$ redis-cli ping
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com0tag:blogger.com,1999:blog-1309977514824160221.post-64061811998851940022016-06-29T06:17:00.002-07:002016-06-29T06:31:22.721-07:00Install and run Redis on Mac OS - Redis Server Installation and First Run<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
Following is the recipe to install Redis server on Mac OS using homebrew in a very simple manner:<br />
<br />
<pre 15="" class="brush: bash" highlight:=""> $ brew install redis
</pre>
<br />
<br />
After installation, you may edit Redis configuration file by going to the path via Mac OS terminal:<br />
<br /></div>
<pre 15="" class="brush: bash" highlight:="">$ /usr/local/etc/redis.conf
</pre>
<br />
To start Redis Server using configuration file on Mac OS, use following command:
<br />
<pre 15="" class="brush: bash" highlight:="">$ redis-server /usr/local/etc/redis.conf
</pre>
<br />
After running the Redis Server on Mac OS, you can check whether Redis Server is running by opening a new terminal window and enter following command:
<br />
<pre 15="" class="brush: bash" highlight:="">$ redis-cli ping
</pre>
The server should reply PONG which means that the server is pinging.<br />
<br /></div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1tag:blogger.com,1999:blog-1309977514824160221.post-73355464811010636862015-02-24T06:04:00.001-08:002015-02-24T09:08:15.386-08:00MongoDB Correct way to start MongoDB - Error Cannot connect to mongodb errno:61 Connection refused [Solved]<div dir="ltr" style="text-align: left;" trbidi="on">
Recently, while involved in full stack Django development and developing an application using Mongodb as database backend, I encountered following error after installing Mongodb on my mac.
Ok first, I installed Mongodb by using brew install command:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">brew update
brew install mongodb
</pre>
<br />
After installing Mongodb, I tried to access the mongo shell by using mongo command, but I got "Cannot connect to mongodb errno:61 Connection refused error" . Here is the full trace:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">alis-air:~ aliraza$ mongo
MongoDB shell version: 2.6.6
connecting to: test
2015-02-24T17:51:36.067+0500 warning: Failed to connect to 127.0.0.1:27017, reason: errno:61 Connection refused
2015-02-24T17:51:36.069+0500 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed
</pre>
<br />
I searched for correct ways to start Mongodb on Mac OS. I tried several solutions but the one which worked for me is as follows:
In the shell write sudo mongod to start it.<br />
<br />
<pre 15="" class="brush: bash" highlight:="">
$ mkdir -p /data/db
$ sudo mongod
</pre>
<br />
This started my Mongodb. Next I opened another shell instance (new shell window) and wrote mongo command and it worked without Error Cannot connect to mongodb errno:61 Connection refused! MongoDB starting Problem Solved.<br />
<br />
<pre 15="" class="brush: bash" highlight:="">$ mongo
MongoDB shell version: 2.6.6
connecting to: test
Server has startup warnings:
2015-02-24T18:20:51.384+0500 [initandlisten]
2015-02-24T18:20:51.384+0500 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
>
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com6tag:blogger.com,1999:blog-1309977514824160221.post-75064736779536640902015-01-30T23:00:00.000-08:002015-01-30T23:09:51.393-08:00Django Setting DEBUG = False causes 500 Error - Django 500 error debug false - ALLOWED_HOSTS Problem [SOLVED]<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYLbxDCMFIAt7u-f8GNv-txVd5wvUtHxn6HrYKBuXRqzwzyiRcxxx1RYPKu4uTFF5uGWJnO_vE6ObTgu_m4K_jfaqGDKbOwFHitukkK0vFZuASuKJQFPhjl8mEWq6g7jnvr2LaulJA-C4/s1600/DjangoPython.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="Python Django alirazabhayani Full stack development" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYLbxDCMFIAt7u-f8GNv-txVd5wvUtHxn6HrYKBuXRqzwzyiRcxxx1RYPKu4uTFF5uGWJnO_vE6ObTgu_m4K_jfaqGDKbOwFHitukkK0vFZuASuKJQFPhjl8mEWq6g7jnvr2LaulJA-C4/s1600/DjangoPython.jpg" title="" /></a></div>
<b>Django Debug False 500 error Solved</b><br />
Being involved in Full Stack Django/Python Web Application Development for 3 years now, I recently deployed my Django application on a test server <span style="font-size: small;"><span style="font-weight: normal;">and encountered Server Error (500) on Django when template debug is set to False. That is in Django settings.py, when I changed DEBUG = FALSE from DEBUG = TRUE, it caused 500 error.</span></span><br />
<br />
<span style="font-size: small;"><span style="font-weight: normal;">I had upgraded my application from Django 1.4 to Django 1.7. I came to know that 500 error on DEBUG = False error is encountered by everyone using django 1.5 or greater as soon as they change their settings.py to DEBUG = False. Well my problem got solved after reading several blogs and digging deeper in the <a href="https://docs.djangoproject.com/en/1.8/releases/1.5/#allowed-hosts-required-in-production" target="_blank">Django Docs for DEBUG = False ALLOWED_HOSTS settings</a>. </span></span><br />
<br />
<span style="font-size: small;"><span style="font-weight: normal;">What I did was, in settings.py, changed ALLOWED_HOSTS value in the following way:</span></span><br />
<pre 15="" class="brush: bash" highlight:="">
ALLOWED_HOSTS = ['123.123.198.123']
# Above mentioned IP is not my actual IP. Enter your actual server IP or domain name here.
</pre>
<br />
If you have a domain name of your server, you may write:
<br />
<pre 15="" class="brush: bash" highlight:="">
ALLOWED_HOSTS = ['www.example.com']
</pre>
You can also use a '*' wildcard to allow all hosts. But this is not recommended in the production environment.
<br />
<pre 15="" class="brush: bash" highlight:="">
ALLOWED_HOSTS = ['*']
</pre>
You can find a full detail regarding the ALLOWED_HOSTS settings here <a href="https://docs.djangoproject.com/en/1.8/ref/settings/#std:setting-ALLOWED_HOSTS" target="_blank">hidden deep inside django document </a>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com21tag:blogger.com,1999:blog-1309977514824160221.post-26719402358845444662015-01-10T04:35:00.000-08:002015-01-10T04:35:26.567-08:00Django Syncdb Create superuser ValueError: unknown locale: UTF-8 [Solved]<div dir="ltr" style="text-align: left;" trbidi="on">
While setting up a new remote server for a new Django Application I am currently working on, I encountered following error as entered yes to create superuser after I entered syncdb command for the first time. </div>
<pre 15="" class="brush: bash" highlight:="">
ValueError: unknown locale: UTF-8
</pre>
Here is the full trace:
<pre 15="" class="brush: bash" highlight:="">
(django1.5)[root@DXDS2102 testdjango]# python manage.py syncdb
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
ValueError: unknown locale: UTF-8
</pre>
As I had centOS server for my Django Application, I entered following commands which actually solved this problem:
<pre 15="" class="brush: bash" highlight:="">
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
</pre>
After running the above command I again did syncdb but Django didn't asked me about creating superuser. So for manually creating a superuser, I entered following command:
<pre 15="" class="brush: bash" highlight:="">
python manage.py createsuperuser
</pre>
The django superuser was created without the error of "ValueError: unknown locale: UTF-8".
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com0tag:blogger.com,1999:blog-1309977514824160221.post-46840284211099983602015-01-06T07:32:00.001-08:002015-01-06T07:32:22.999-08:00django.core.exceptions.improperlyconfigured error loading mysqldb module no module named mysqldb - Python Django MySQL Error<div dir="ltr" style="text-align: left;" trbidi="on">
In order to deal massive (Big) data in one of my applications. I am testing MySQL with MariaDB and TokuDB. Previously I had used PostgreSQL for my applications but the Reporting Engine I am designing requires some unique features to update data faster and retrieve it in an efficient manner. TokuDB being based on Fractal Indexing makes the insertion faster for Bigger Data and keeps the database compressed resulting in efficient use of storage data. While running MySQL MariaDB with Django in my virtual environment, I encountered following error: <br />
<br />
<br />
<pre 15="" class="brush: bash" highlight:="">
django.core.exceptions.improperlyconfigured error loading mysqldb module no module named mysqldb
</pre>
Full Trace is:
<br />
<pre 15="" class="brush: bash" highlight:="">
(django1.5)alis-air:DjangoMariaDb aliraza$ python manage.py syncdb
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
utility.execute()
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/__init__.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/__init__.py", line 77, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 8, in <module>
from django.core.management.sql import custom_sql_for_model, emit_post_sync_signal
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/core/management/sql.py", line 9, in <module>
from django.db import models
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/db/__init__.py", line 40, in <module>
backend = load_backend(connection.settings_dict['ENGINE'])
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/db/__init__.py", line 34, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/db/utils.py", line 93, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/db/utils.py", line 27, in load_backend
return import_module('.base', backend_name)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 17, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
</pre>
The solution to this error which worked for me was simply installing mysql-python module in my virtual environment. Use following command:
<br />
<pre 15="" class="brush: bash" highlight:="">
pip install mysql-python
</pre>
and if you are not using virtualenv, then use:
<br />
<pre 15="" class="brush: bash" highlight:="">
sudo pip install mysql-python
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1tag:blogger.com,1999:blog-1309977514824160221.post-31135442307643971062014-12-24T06:11:00.000-08:002014-12-24T06:13:35.778-08:00Psycopg2 - MacOS X - library not loaded libssl.1.0.0.dylib reason image not found - Issue Resolved <div dir="ltr" style="text-align: left;" trbidi="on">
I encountered following error on my MacOS X after installing PostgreSQL and Psycopg2 for my Python and Django based application.<br />
<br />
<pre 15="" class="brush: bash" highlight:="">ImportError: dlopen(/Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/psycopg2/_psycopg.so, 2): Library not loaded: libssl.1.0.0.dylib
Referenced from: /Users/aliraza/Projects/VirtualEnv/django1.5/lib/python2.7/site-packages/psycopg2/_psycopg.so
Reason: image not found</pre>
<br />
<b>STEP-1: </b><br />
Check if you have openssl installed by writing openssl on the commandline:<br />
<pre 15="" class="brush: bash" highlight:="">
$ openssl
</pre>
<br />
If it starts, this means its installed in your system and you may skip STEP-2 and go directly to STEP-3.<br />
<br />
<b>STEP-2: </b><br />
If openssl isn't installed, install it by:<br />
<pre 15="" class="brush: bash" highlight:="">
$ brew install openssl
</pre>
<br />
<b>STEP-3: </b><br />
Check the path of openssl lib on your machine. On my machine it was:<br />
<pre 15="" class="brush: bash" highlight:="">
$ /usr/local/opt/openssl/lib/
</pre>
<br />
<b>STEP-4:</b><br />
You have to simply include this path from STEP-3 by running following command:<br />
<pre 15="" class="brush: bash" highlight:="">
$ export DYLD_LIBRARY_PATH=***YOUR OPENSSL PATH FROM STEP-3 HERE***
</pre>
i.e in my case, the command with the STEP-3 path is:<br />
<pre 15="" class="brush: bash" highlight:="">
$ sudo export DYLD_LIBRARY_PATH=/usr/local/opt/openssl/lib/
</pre>
<br />
You may also have to create a symbolic link to PostgreSQL's lib folder. For that, first check your path of PostgreSQL's lib folder and then write following commands accordingly:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">
$ sudo ln -s /Library/PostgreSQL/9.4/lib/libssl.1.0.0.dylib /usr/lib/
$ sudo ln -s /Library/PostgreSQL/9.4/lib/libcrypto.1.0.0.dylib /usr/lib/
</pre>
<br />
Now run python/ Django shell and import psycopg2. I had no errors after following this recipe. </div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1tag:blogger.com,1999:blog-1309977514824160221.post-63551554602473173312014-03-25T08:31:00.002-07:002014-03-25T08:31:29.049-07:00PostgreSQL - pg_dump to copy PostgreSQL Database From One Remote Server to Another<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
While involved in Python + Django based projects and using postgreSQL as a database backend, there are occasions where I need to switch between servers and at times need to copy databases as well. For transferring the django application data + schema from one remote server to another, I use following ways as per convenience, available options, server restrictions etc.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
<b>1)</b> Copying Django application data + schema (PostgreSQL) from one remote server by dumping the data in a file and then transferring that file to another remote server and restoring it. In this approach, you have got three options:</div>
<div style="text-align: left;">
<br /></div>
<ul style="text-align: left;">
<li>Dump both schema as well as Data to the file:</li>
</ul>
<pre 15="" class="brush: bash" highlight:="" style="text-align: left;">pg_dump -C -h host -U username db_name > /any_directory/dump_schema_and_data_file
</pre>
<ul style="text-align: left;">
<li>Dump only schema from PostgreSQL Database to a file:</li>
</ul>
<pre 15="" class="brush: bash" highlight:="" style="text-align: left;">pg_dump -Cs -h host -U username db_name > /any_directory/dump_schema_file
</pre>
<ul style="text-align: left;">
<li>Dump only data from PostgreSQL Database to file:</li>
</ul>
<pre 15="" class="brush: bash" highlight:="" style="text-align: left;">pg_dump -a -h host -U username db_name > /any_directory/dump_data_file
</pre>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
After copying the above file to target server, you can easily restore database from file by using following command:<br />
</div>
<pre 15="" class="brush: bash" highlight:="" style="text-align: left;">psql -h host -U username db_name < data_schema_file
</pre>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: justify;">
<b>2)</b> Second approach I use to copy PostgreSQL database from one remote server to another is by directly using following command i.e without copying the data to intermediate file:</div>
<div style="text-align: left;">
<br /></div>
<pre 15="" class="brush: bash" highlight:="" style="text-align: left;">pg_dump -C -h localhost -U localuser db_name | psql remotehost -U remoteuser db_name
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com6tag:blogger.com,1999:blog-1309977514824160221.post-31280610183510061442013-07-04T00:45:00.002-07:002013-07-04T00:59:13.495-07:00Django - Install Psycopg2 on Mac OS X - pg_config path issue AND lipo: can't open input file - Issues Resolved<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
In one of my recent Django based projects, I happened to use PostgreSQL as Database Backend. In order to connect the db and for other operations with PostgreSQL, python's psycopg2 module is pretty popular and readily used. Psycopg2 got easily installed on the Linux (Ubuntu) machine but I faced some issues when I tried to install it on my Mac OS X 10.6.7. After following several tips and reading several blog posts, I finally managed to find the actual working solution which resolved the issues of installing Psycopg2 on Mac OS X. I also noticed that many people were trying to find solution of the same issues which I faced. So I am writing their solutions in a single blog post in a descriptive and step-by-step manner. There are basically two most common issues which I also faced while installing psycopg2 on Mac OS X:</div>
<br />
<br />
<u><b>PROBLEM 1:</b></u><br />
<br />
When I tried to install psycopg2 on Mac OS X and entered following command on my terminal:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">>pip install psycopg2</pre>
<br />
I got an error that pg_config file's path is not specified. Following is the full trace:<br />
<br />
TRACE:<br />
<pre 15="" class="brush: bash" highlight:="">Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build/psycopg2
</pre>
<br />
<br />
<u><b>SOLUTION OF PROBLEM 1</b>:</u><br />
<br />
<div style="text-align: justify;">
Provided that some version of PostgreSQL is already installed, you will have to explicitly add the PATH of pg_config file in your PostgreSQL bin folder. In my case, the installation PATH for PostgreSQL is:</div>
<br />
<pre 15="" class="brush: bash" highlight:=""> /opt/local/lib/postgresql91/
</pre>
<br />
so to define the PATH of pg_config file, I entered following command in my terminal:<br />
<pre 15="" class="brush: bash" highlight:="">> PATH=$PATH:/opt/local/lib/postgresql91/bin/
</pre>
<br />
After running this command, the pg_config path error while installing psycopg2 was resolved<br />
<br />
<br />
<u><b>PROBLEM 2:</b></u><br />
<br />
While installing psycopg2 on my Mac OS X, the pip install psycopg2 command returned following error:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">lipo: can't open input file: /var/tmp//ccTy5xAu.out (No such file or directory)
</pre>
<br />
Full trace of the error is as follows:<br />
<br />
<pre 15="" class="brush: bash" highlight:="">psycopg/psycopgmodule.c:951: fatal error: error writing to -: Broken pipe
compilation terminated.
lipo: can't open input file: /var/tmp//ccTy5xAu.out (No such file or directory)
error: command 'gcc-4.2' failed with exit status 1
</pre>
<br />
<br />
<u><b>SOLUTION OF PROBLEM 2:</b></u><br />
<br />
<div style="text-align: justify;">
After a googling a bit, and trying several recipes, the solution that worked for me was the following command which overrides the ARCHFLAGS architecture settings:</div>
<br />
<pre 15="" class="brush: bash" highlight:="">sudo env ARCHFLAGS="-arch i386 -arch x86_64" pip install psycopg2
</pre>
<br />
This command finally installed psycopg2 on my Mac OS X without any issues.</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com14tag:blogger.com,1999:blog-1309977514824160221.post-11688951880820832552013-02-04T21:53:00.000-08:002013-02-05T00:38:50.739-08:00Django Deployment - Fabric, GUnicorn, NGInx, Supervisor - Django-fagungis<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
Django has become a very popular web framework and I see lots of people confused about deploying Django projects the right way using the right tools. I myself faced a lot of problems in finding the right combination of tools for successful (and hassle free) Django Deployment.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Previously, I used Apache and mod_wsgi for Django Deployment but it required a lots of tweaking and found it to be a very lengthy process. You may go through <a href="http://rdegges.com/deploying-django" target="_blank">this post</a> to read whole account on Django Deployment issues using Apache and mod_wsgi.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
After several cups of coffee and reading a lots of blogs, I found some alternatives/new-ways for Django Deployment. Those new ways/tools involved <a href="http://wiki.nginx.org/Main" target="_blank">Nginx</a>, <a href="http://gunicorn.org/" target="_blank">gunicorn</a> and <a href="http://supervisord.org/" target="_blank">supervisord</a> with their lots of configurations. Still, when I started to use them, I found them more friendly for Django Deployment. But still I had to make some project related adjustments every time I used to deploy a new production-ready version of my project. Then I found this great tool "django-fagungis" (<a href="https://github.com/DNX/django-fagungis" target="_blank">git</a> - <a href="https://bitbucket.org/DNX/django-fagungis/" target="_blank">bitbucket</a>) which promises a three step hassle free Django Deployment. After configuring it once, I just have to literally type three commands and its does my Django Deployment everytime like Aladdin's genie would have done for Aladdin. From cloning my project directly from bitbucket to configuring Nginx, Gunicorn and Supervisord on my remote server, it handles it all like a charm. You can find a detailed tutorial on how to use django-fagungis on its <a href="https://bitbucket.org/DNX/django-fagungis/" target="_blank">official site</a>. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So, after following the initial configuration instructions of django-fagungis, I now have to type just three commands from my local machine!</div>
<br />
Step1:<br />
<pre 15="" class="brush: bash" highlight:="">fab my_project test_configuration
</pre>
<br />
Step2:<br />
<pre 15="" class="brush: bash" highlight:="">fab my_project setup
</pre>
<br />
Step3:<br />
<pre 15="" class="brush: bash" highlight:="">fab my_project deploy
</pre>
</div>
Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1Karachi, Pakistan24.893379 67.02806090000001424.4328985 66.38261390000001 25.3538595 67.673507900000018tag:blogger.com,1999:blog-1309977514824160221.post-14570018398293554942012-02-12T08:50:00.000-08:002012-02-12T09:50:41.207-08:00Django Deployment - Setup mod_wsgi on CentOS<div dir="ltr" style="text-align: left;" trbidi="on">Following terminal command would install mod_wsgi on CentOS:<br />
<br />
<pre class="brush: bash" highlight: [5, 15]>sudo yum install mod_wsgi
yum list installed mod_wsgi
</pre><br />
</div>Ali Raza Bhayanihttp://www.blogger.com/profile/01886806921103822404noreply@blogger.com1